Many goal-oriented dialog tasks, especially ones in which the dialog system has to interact with external knowledge sources such as databases, have to handle a large number of Named Entities (NEs). There are at least two challenges in handling NEs using neural methods in such settings: individual NEs may occur only rarely making it hard to learn good representations of them, and many of the Out Of Vocabulary words that occur during test time may be NEs. Thus, the need to interact well with these NEs has emerged as a serious challenge to building neural methods for goal-oriented dialog tasks. In this paper, we propose a new neural method for this problem, and present empirical evaluations on a structured Question Answering task and three related goal-oriented dialog tasks that show that our proposed method can be effective in interacting with NEs in these settings.
The paper addresses the task of dealing with named entities in goal oriented dialog systems. Named entities, and rare words in general, are indeed troublesome since adding them to the dictionary is expensive, replacing them with coarse labels (ne_loc, unk) looses information, and so on. The proposed solution is to extend neural dialog models by introducing a named entity table, instantiated on the fly, where the keys are distributed representations of the dialog context and the values are the named entities themselves. The approach is applied to settings involving interacting to a database and a mechanism for handling the interaction is proposed. The resulting model is illustrated on a few goal-oriented dialog tasks.
I found the paper difficult to read. The concrete mappings used to create the NE keys and attention keys are missing. Providing more structure to the text would also be useful vs. long, wordy paragraphs. Here are some specific questions:
1. How are the keys generated? That are the functions used? Does the "knowledge of the current user utterance" include the word itself? The authors should include the exact model specification, including for the HRED model.
2. According to the description, referring to an existing named entity must be done by "generating a key to match the keys in the NE table and then retrieve the corresponding value and use it". Is there a guarantee that a same named entity, appearing later in the dialog, will be given the same key?  Or are the keys for already found entities retrieved directly, by value?
3. In the decoding phase, how does the system decide whether to query the DB?
4. How is the model trained?
In its current form, it's not clear how the proposed approach tackles the shortcomings mentioned in the introduction. Furthermore, while the highlighted contribution is the named entity table, it is always used in conjunction to the database approach. This raises the question whether the named entity table can only work in this context.
For the structured QA task, there are 400 training examples, and 100 named entities. This means that the number of training examples per named entity is very small. Is that correct? If yes, then it's not very surprising that adding the named entities to the vocabulary leads to overfitting. Have you compared with using random embeddings for the named entities?
Typos: page 2, second-to-last paragraph: firs -> first, page 7, second to last paragraph: and and -> and