Domain specific goal-oriented dialogue systems typically require modeling three types of inputs, viz., (i) the knowledge-base associated with the domain, (ii) the history of the conversation, which is a sequence of utterances and (iii) the current utterance for which the response needs to be generated. While modeling these inputs, current state-of-the-art models such as Mem2Seq typically ignore the rich structure inherent in the knowledge graph and the sentences in the conversation context. Inspired by the recent success of structure-aware Graph Convolutional Networks (GCNs) for various NLP tasks such as machine translation, semantic role labeling and document dating, we propose a memory augmented GCN for goal-oriented dialogues. Our model exploits (i) the entity relation graph in a knowledge-base and (ii) the dependency graph associated with an utterance to compute richer representations for words and entities. Further, we take cognizance of the fact that in certain situations, such as, when the conversation is in a code-mixed language, dependency parsers may not be available. We show that in such situations we could use the global word co-occurrence graph and use it to enrich the representations of utterances. We experiment with the modified DSTC2 dataset and its recently released code-mixed versions in four languages and show that our method outperforms existing state-of-the-art methods, using a wide range of evaluation metrics.
The current paper proposes using Graph Convolutional Networks (GCN) to explicitly represent and use relational data in dialog modeling, as well an attention mechanism for combining information from multiple sources (dialog history, knowledge base, current utterance). The work assumes that the knowledge base associated with the dialog task has en entity-to-entity-relationship format and can be naturally expressed as a graph. The dependency tree of dialog utterances can also be expressed as a graph, and the dialog history as a set of graphs. To utilize this structure, the proposed method uses GCNs whose lowest layer embeddings are initialized with the entity embeddings or via outputs of standard RNN-like models. The main claim is that the proposed model outperforms the current state-of-the-art on a goal-oriented dialog task.
The idea of explicitly modeling the relational structure via GCNs is interesting. However, the use of GCNs independently per sentence and per knowledge-base is a bit disappointing, since it does not couple these sources of information in a structured way. Instead, from my current understanding, the approach merely obtains better representations for each of these sources of information, in the same way it is done in the related language tasks. For instance, have you considered passing information across the trees in the history as well? Or aligning the parsed query elements with the KB elements?
The results are very good. That said, a source of concern is that the model is only evaluated as a whole, without showing which modification brought the improvements. The comparison between using/not using RNNs to initiate the first GCN layer is promising, but why not compare to using only RNN also? Why not compare the various encoders within an established framework (e.g. without the newly introduced attention mechanism)? Finally, the attention mechanism, stated as a contribution, is not motivated well.
Clarity:
The notation is described well, but it's not terribly intuitive (the query embedding is denoted by c, the history embedding by a, etc.), making section 4.4. hard to follow. A figure would have made things easier to follow, esp. due to the complexity of the model. A clearer parallel with previous methods would also improve the paper: is the proposed approach adding GCN on top of an established pipeline? Why not?
More discussion on code-mixed language, e.g. in section 4.6, would also improve clarity a bit (make the paper more self-contained). While the concept is clear from the context, it would be helpful to describe the level of structure in the mixed language. For instance, can dependency trees not be obtained code-mixed languages? Is there any research in this direction? (or is the concept very new?) Maybe I am just missing the background here, but it seems helpful in order to asses how appropriate the selected heuristic (based on the co-occurence matrix) is.
Relevant Reference:
Learning Graphical State Transitions, Johnson, ICLR 2017 also uses graph representations in question answering, though in a somewhat different setting.
Typos:
Section 4: "a model with following components"
Section 5: "the various hyperparameters that we conisdered"