We propose an adversarial learning approach to the generation of multi-turn dialogue responses. Our proposed framework, hredGAN, is based on conditional generative adversarial networks (GANs). The GAN's generator is a modified hierarchical recurrent encoder-decoder network (HRED) and the discriminator is a word-level bidirectional RNN that shares context and word embedding with the generator. During inference, noise samples conditioned on the dialogue history are used to perturb the generator's latent space to generate several possible responses. The final response is the one ranked best by the discriminator. The hredGAN shows major advantages over existing methods: (1) it generalizes better than networks trained using only the log-likelihood criterion, and (2) it generates longer, more informative and more diverse responses with high utterance and topic relevance even with limited training data. This superiority is demonstrated on the Movie triples and Ubuntu dialogue datasets with both the automatic and human evaluations.
The paper applies conditional GAN to the HRED model [Serban et al., 2016] for dialogue response generation, showing improvements in terms of informativeness and diversity compared to HRED and VHRED [Serban et al., 2017].
The paper is technically solid and relatively easy to follow and the results are good, but comparisons with previous work (descriptive and experimental) are rather weak. 
- Related work is incomplete: The paper specifically argues for the use of GAN to improve diversity in dialogue response generation, but this is not the first paper to do so. [Xu et al., 2017] presents a GAN-like setup that targets exactly the same goal, but that work is not cited in the paper. Same for [Zhang et al., 2018], but the latter work is rather recent (it still should probably be cited).
- Evaluation: There is no evaluation against Xu et al., which targets the same goal. The authors didn’t even compare their methods against baselines used in other GAN works for diverse response generation (e.g., MMI [Xu et al.; Zhang et al.], Li et al.’s GAN approach [Xu et al.]), which makes it difficult to draw comparisons between these related methods. As opposed to these other works, the paper doesn’t offer any human evaluation.
- It would have been nice to include an LSTM or GRU baseline, as these models are still often used in practice and the VHRED paper suggests [Serban et al., 2016; Table 1] that LSTM holds up quite well against HRED (if we extrapolate the results of VHRED vs. LSTM and VHRED vs. HRED). The ablation of GAN and HRED would help us understand which of the two is more important.
In sum, the work is relatively solid, but considering how much has already been done on generating diverse responses (including 3 other papers also using GAN), I don’t think this paper is too influential. Its main weakness is the evaluation (particularly the lack of human evaluation.)
Minor comments:
- Introduction: “diversity promoting training objective but their model is for single turn conversations”. 
If these were “single turns”, they wouldn’t really be called conversations; that objective has been used with 3+ turn conversations. It can actually be applied to multi-turn dialogue as with any autoegressive generative models. For example, it has been exploited that way as a baseline for multi-turn dialogue [Li et al. 2016](“Deep Reinforcement Learning for Dialogue Generation“). Note it is not a “training objective”, but only an objective function at inference time, which is a more valid reason to criticize that paper.
- “We use greedy decoding (MLE) on the first part of the objective.” Doesn’t that hurt diversity because of MLE? what about using sampling instead (maybe with temperature)?
- Algorithm 1: the P_theta_G don’t seem to match the text of section 2. h_i is in sometimes written in bold and sometimes not (see also Eq 12 for comparison.)
- End of section 2.1: There are multiple Li et al.; specify which one.
- End of section 2.2 and 2.4: extra closing parenthesis after N(0, …))
- Figures are too small to read the subscripts.
[Xu et al. 2017]: Zhen Xu, Bingquan Liu, Baoxun Wang, Sun Chengjie, Xiaolong Wang, Zhuoran Wang, and Chao Qi. Neural response generation via gan with an approximate embedding layer. EMNLP 2017.
[Zhang et al. 2018]: Zhang, Yizhe & Galley, Michel & Gao, Jianfeng & Gan, Zhe & Li, Xiujun & Brockett, Chris & Dolan, Bill. (2018). Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization.