Building chatbots that can accomplish goals such as booking a flight ticket is an unsolved problem in natural language understanding. Much progress has been made to build conversation models using techniques such as sequence2sequence modeling. One challenge in applying such techniques to building goal-oriented conversation models is that maximum likelihood-based models are not optimized toward accomplishing goals. Recently, many methods have been proposed to address this issue by optimizing a reward that contains task status or outcome. However, adding the reward optimization on the fly usually provides little guidance for language construction and the conversation model soon becomes decoupled from the language model. In this paper, we propose a new setting in goal-oriented dialogue system to tighten the gap between these two aspects by enforcing model level information isolation on individual models between two agents. Language construction now becomes an important part in reward optimization since it is the only way information can be exchanged. We experimented our models using self-play and results showed that our method not only beat the baseline sequence2sequence model in rewards but can also generate human-readable meaningful conversations of comparable quality.
I like the idea of coupling the language and the conversation model. This is in line with the latest trends of constructing end-to-end NN models that deal with the conversation in a holistic manner. The idea of enforcing information isolation is brilliant. Creating hidden information and allowing the two-party model to learn through self-play is a very interesting approach and the results seem promising. 
Having said that, I feel important references are missing and specific statements of the paper, like that "Their success is however limited to conversations with very few turns and without goals" can be argued. There are papers that are goal oriented and have many turns. I will just provide one example, to avoid being overwhelming, although more can be found in the literature. That would be the paper of T.-H. Wen, D. Vandyke, N. Mrksic, M. Gasic, L. Rojas-Barahona, P.-H. Su, S. Ultes and S. Young (2017). "A Network-based End-to-End Trainable Task-oriented Dialogue System." EACL 2017, Valencia, Spain. In fact in this paper even more dialogue modules are coupled. So, the "fresh challenge" of the paper can be argued. 
It is not clear to me how you did the supervised part of the training. To my experience, although supervised learning can be used, reinforcement learning seems to be the most popular choice. Also, I had to read most of the paper to understand that the system is based on a simulator. Additionally, it is not clear how you got the ground-truth for the training. How are the action and the dialogue generated by the simulator guaranteed to follow the optimal policy?
 
I also disagree with the statement that "based on those... to estimate rewards". If ruled-based systems were sufficient, there would not be a need for statistical dialogue managers. However, the latter is a very active research area. 
Figure 1 is missing information (for my likings), like not defined symbols. In addition, it's not self-contained. Also, I would prefer a longer, descriptive and informative label to make the figure as self-explained as possible. I believe it would add to the clarity of the paper. 
Also, fundamental information, according to my opinion is missing. For example, what are the restrictions R and how is the database K formed? What is the size of the database? How many actions do you define? Some of them are defined in the action state decoder, but it is not clear if it is all of them.
GRU -> abbreviation not defined
I would really appreciate a figure to better explain the subsection "Encoding External Knowledge". In the current form I am struggling to understand what the authors mean. 
How is the embedding matrix E created?
Have you tried different unit sizes d? Have you tried different unit sizes for the customer and the service? 
"we use 2 transformation matrixes" -> if you could please provide more details
How is equation 2 related to figure 1? 
Typo: "name of he person"
"During the supervised learning... and the action states". I am not sure I get what you mean. May you make this statement more clearly by adding an equation for example?
What happens if you use random rather than supervised learning weight initialisation? 
Equation 7: What does T stand for?
I cannot find Table 1, 2 and 5 referred to in-text. Moreover I am not sure about quite some items. For example, what is number db? What is the inference set? 
500k of data is quite some. A figure on convergence would be nice.
Setting generator: You mention the percentage of book and flight not found. What about the rest of the cases? 
Typo: “table 3 and table 3”
The set of the final states of the dialogue is not the same as the ones presented at Fig. 2.
Sub section reward generation is poorly described. After all, reward seems to play a very important role for the proposed system. Statements like  “things such as” (instead the exhaustive list of rules for example) or “the error against the optimal distance” with no note what should be considered the optimal distance make the paper clarity decreased and the results not possible to be reproduced. Personally I would prefer to see some equations or a flow chart. By the way, have you tried and alternative reward function? 
Table 4 is not easy for me to understand. For example, what do you mean when you say eval reward?
Implementation details. I fail to understand how the supervised learning is used (as said already). Also you make a note for the value network, but not for the policy network. 
There are some minor issues with the references such as pomdp or lstm not being capitalised
In general, I believe that the paper has a great potential and is a noticeable work. However, 
the paper could be better organised. Personally, I struggled with the clarity of some text portions.
For me, the main drawback of the paper is that it was't tested with human users. The actual success of the system when evaluated by humans can be surprisingly different from the one that comes from simulation.