Generic responses are a known issue for open-domain dialog generation. Most current approaches model this one-to-many task as a one-to-one task, hence being unable to integrate information from multiple semantically similar valid responses of a prompt. We propose a novel dialog generation model that learns a semantic latent space, on which representations of semantically related sentences are close to each other. This latent space is learned by maximizing correlation between the features extracted from prompt and responses. Learning the pair relationship between the prompts and responses as a regression task on the latent space, instead of classification on the vocabulary using MLE loss, enables our model to view semantically related responses collectively. An additional autoencoder is trained, for recovering the full sentence from the latent space. Experimental results show that our proposed model eliminates the generic response problem, while achieving comparable or better coherence compared to baselines.
1. Summary: The authors proposed to alleviate the generic response problem in open-domain dialog generation by generating a response to a prompt from a semantic latent vector. This vector needs to be located close to the latent vector of the corresponding prompt. To this end, they employ canonical correlation analysis and auto-encoder to learn the mapping from a sequence of text to a semantic latent vector. To model the variations and topic shifts that may happen in responses, they also use a separate intermediate vector in the auto-encoder of generating responses.
2. Overall assessment: While this paper is quite fun to read, it is not innovative enough and it lacks some critical experiments and error analysis to be accepted this time. I'll elaborate on these problems in my comments below. 
3. Strengths:
3.1 This paper is well written. The motivation and the main idea are well explained and pretty easy to understand. Although there are some details missing, it doesn't prevent readers from understanding and enjoying this paper.
3.2 The use of human evaluation is a great plus. Subtle characteristics of text, such as readability, coherence, and specificity cannot be well justified by automatic evaluation. Human evaluation is a must for these aspects. It's great to see the authors include this in this paper.
4. Weakness and questions:
4.1 It seems the model in this paper can be connected to adversarial learning from some angles. It would be great to see such analysis in this paper.
4.2 The experiments in this paper do not look thorough enough. There are many more things should be included, such as error analysis, comparisons over different variations of the proposed model and so on. What if we remove $Y_u$ in both training and inference but keep the overall model the same? How important is $Y_u$ and what's its effect? Many such questions haven't been answered in this paper.
4.3 Embedding average cosine similarity seems to be too simple to be used for evaluation as it omits the contextual information and semantic meaning of a sentence.
4.4 It would be interesting to see the Dist-1 and Dist-2 scores of the gold standard responses. It's a good reference to let us know ho diversified the real responses are.
4.5 I'd like to see some explanation of how the raters rate specificity and coherence. What are the standards they use? What are the instructions the author give to them?
4.6 Are improvements significant in Table 1? It seems the proposed model is not performing very stably over different metrics. Some anlysis on this would be great.