Although recent neural conversation models have shown great potential, they often generate bland and generic responses. While various approaches have been explored to diversify the output of the conversation model, the improvement often comes at the cost of decreased relevance (Zhang et al., 2018). In this paper, we propose a SPACEFUSION model to jointly optimize diversity and relevance that essentially fuses the latent space of a sequence-to-sequence model and that of an autoencoder model by leveraging novel regularization terms. As a result, our approach induces a latent space in which the distance and direction from the predicted response vector roughly match the relevance and diversity, respectively. This property also lends itself well to an intuitive visualization of the latent space. Both automatic and human evaluation results demonstrate that the proposed approach brings significant improvement compared to strong baselines in both diversity and relevance. 1 1 Introduction The field of neural response generation is advancing rapidly both in terms of research and commercial applications (Gao et al., 2019; Zhou et al., 2018; Yoshino et al., 2019; Zhang et al., 2019). Nevertheless, vanilla sequence-to-sequence (S2S) models often generate bland and generic responses (Li et al., 2016a). Li et al. (2016a) encourage diversity by re-ranking the beam search results according to their mutual information with the conversation context. However, as beam search itself often produces lists of nearly identical sequences, this method can require a large beam width (e.g. 200). As a result, re-ranking can be extremely 1 An implementation of our model is available at https: //github.com/golsun/SpaceFusion 2 For simplicity, we omitted the response at the center: I would love to play this game. This highlights the need to improve the diversity of candidates before re-ranking, and the need to optimize for diversity during training rather than just at the decoding stage. While various approaches have been explored to diversify the output of conversation models, the improvement often comes at the cost of decreased response relevance along other dimensions. For instance, Zhao et al. (2017) present an approach to enhancing diversity by mapping diverse responses to a probability distribution using a conditional variational autoencoder (CVAE). Despite the improved response diversity, this approach reduces response relevance as measured against the baseline. One possible reason for this diversity-relevance trade-off is that such probabilistic approaches are not explicitly encouraged to induce a disentangled representation in latent space for controlling diversity and relevance independently. Consider a Gaussian distribution, which is widely used for CVAE. A Gaussian distribution naturally brings frequent responses near its mean, and the resulting responses are often generic and boring. To generate diverse and interesting responses, one needs to sample a little distance from the mean. But doing so naturally leads to infrequent and thus even irrelevant responses. In this paper, we propose a novel geometrical approach that explicitly encourages a structured latent space in which the distance and direction from a predicted response vector roughly match the relevance and diversity, respectively, as illustrated in Figure 1. To induce such a latent space, we leverage two different models: 1) a S2S model, producing the predicted response vector (the black dot at the center in Figure 1), and 2) an autoencoder (AE) model, yielding the vectors for potential responses (the colored dots). In order to make the S2S and AE share the same latent space (the cloud), we use the same decoder for both and train them jointly end-to-end with novel regularization terms. As this fuses the two latent spaces, we refer to our model as SPACEFUSION . Regularization is necessary because only sharing the decoder, as in (Luan et al., 2017), does not necessarily align the latent spaces obtained by S2S and AE respectively or impose a disentangled structure onto the space. We introduce two regularization terms to tackle this issue. 1) interpolation term: we encourage a smooth semantic transition along the path between the predicted response vector and each target response vector (ar-rowed lines in Figure 1). This term effectively prevents semantically different responses from aligning in the same direction, essentially scattering them over different directions. 2) fusion term: we want the vectors from the two models to be distributed in a homogeneous manner, rather than forming two separate clusters (Figure 5) that can potentially make sampling non-trivial. With the resulting latent space, we can control relevance and diversity by respectively adjusting distance and direction from a predicted response vector, without sacrificing each other greatly. Our approach also lends itself well to the intuitive visualization of latent space. Since our model allows us to geometrically find not only the predicted response vector but also the target response vector as in Figure 5, we can visually interpret the structure of latent space and identify major issues thereof. We devote Section 5.1 to show comprehensive examples for visualization-based analysis. Automatic and human evaluations demonstrate that the proposed approach improves both the diversity and relevance of the responses, compared to strong baselines on two datasets with one-to-many context-response mapping. Grounded conversation models utilize extra context inputs besides conversation history, such as persona (Li et al., 2016b), textual knowledge (Ghazvininejad et al., 2017; Galley et al., 2019), dialog act (Zhao et al., 2017) and emotion (Huber et al., 2018). Our approach does not depend on such extra input and thus is complementary to this line of studies. Variational autoencoder (VAE) models explicitly model the uncertainty of responses in latent space. Bowman et al. (2016) used VAE with Long-Short Term Memory (LSTM) cells to generate sentences. The basic idea of VAE is to encode the input x into a probability distribution (e.g. Gaussian) z instead of a point encoding. However, it suffers from the vanishing latent variable problem (Bowman et al., 2016; Zhao et al., 2017) when applied to text generation tasks. Bowman et al. (2016); Fu et al. (2019) proposed to tackle this problem with word dropping and specific KL annealing methods. Zhao et al. (2017) proposed to add a bag-of-word loss, complementary to KL annealing. Applying this to a CVAE conversation model, they showed that even greedy decoding can generate diverse responses. However, as VAE/CVAE conversation models can be limited to a simple latent representations such as standard Gaussian distribution, Gu et al. (2018) proposed to enrich the latent space by leveraging a Gaussian mixture prior. Our work takes a geometrical approach that is fundamentally different from probabilistic approaches to tackle the limitations of parameteric distributions in representation and difficulties in training. Decoding and ranking encourage diversity during the decoding stage. As vanilla beam search often produces lists of nearly identical sequences, Vijayakumar et al. (2016) propose to include a dissimilarity term in the objective of beam search decoding. Li et al. (2016a) re-ranked the results obtained by beam search based on mutual information with the context using a separately trained response-to-context S2S model. Multi-task learning is another line of studies related to the present work (see Section 3.2). Sennrich et al. (2016) use multi-task learning to improve neural machine translation by utilizing monolingual data, which usually far exceeds the amount of parallel data. A similar idea is applied by Luan et al. (2017) to conversational modeling, involving two tasks: 1) a S2S model that learns a context-to-response mapping using conversation data, and 2) an AE model that utilizes speaker-specific non-conversational data. The decoders of S2S and AE were shared, and the two tasks were trained alternately. Let D = [( x 0 , y 0 ) , ( x 1 , y 1 ) ,    , ( x n , y n )] denote a conversational dataset, where x i and y i are a context and its response, respectively. x i consists of one or more utterances. Our aim is to train a model on D to generate relevant and diverse responses given a context. We design our model to induce a latent space where different responses for a given context are in different directions around the predicted response vector, as illustrated in Figure 1. Then we can obtain diverse responses by varying the direction and keep their relevance by sampling near the predicted response vector. To fulfill this goal, we first produce the predicted response representation z S2S and target response representations z AE using an S2S model and an AE model, respectively, as illustrated in Figure 2. Both encoders are implemented using stacked Gated Recurrent Unit (GRU) (Cho et al., 2014) cells followed by a noise layer that adds multivariate Gaussian noise (cid:15)  N (0 ,  2 I ) . We then explicitly encourage smooth semantic transition along the path from z S2S to z AE by imposing any interpolation between them to generate the same response via the following loss term: L interp =  1 | y | log p ( y | z interp ) (1) where z interp = uz S2S + (1  u ) z AE and u  U (0 , 1) is a uniformly distributed random vari-S2S encoder AE encoder decoder decoder  S2  AE  S2  AE  interp decoder  interp   shared parameters  fuse + + Figure 2: SPACEFUSION model architecture. able. | y | is the number of words in y . Note that it is this regularization term that effectively prevents significantly different responses from aligning in the same direction, essentially scattering them over different directions. In order for this interpolation loss to work, we share the same decoder for both AE and S2S models as in (Luan et al., 2017). The decoder consists of stacked GRU cells followed by a softmax layer. It is worth mentioning that z interp is not just randomly drawn from a single line but from a richer probabilistic region as both z interp and z S2S are stochastic due to the random component (cid:15) . Now, we want vectors from both the AE and S2S models to be distributed in a homogeneous manner scattered over the entire space while keeping the distance between z S2S and z AE as small as possible for any (context-response) pair in the training data. This objective is represented in the following regularization term: L fuse = (cid:88) i  batch d ( z S2S ( x i ) , z AE ( y i )) n  (cid:88) i,j  batch ,i (cid:54) = j d ( z S2S ( x i ) , z S2S ( x j )) n 2  n  (cid:88) i,j  batch ,i (cid:54) = j d ( z AE ( y i ) , z AE ( y j )) n 2  n (2) where n is the batch size and d ( a, b ) is the root mean square of the difference between a and b . For each batch, we basically disperse vectors obtained by the same model and pull the predicted response vectors to the corresponding target response vectors. In practice, we found that the performance is better if the Euclidean distance is clipped to a prescribed maximum value. 3 Finally, with weight parameters  and  , the 3 This value is set as 0.3 for the present experiments loss function is defined as: L =  1 | y | log p ( y | z S2S )  1 | y | log p ( y | z AE ) +  L interp +  L fuse (3) As L interp and L fuse encourage the path between z S2S and z AE to be smooth and short while scattering vectors over the entire space, they effectively fuse the z S2S latent space and the z AE latent space. Accordingly we refer this approach as SPACEFUSION with path regularization. In contrast to previous multi-task conversation model (Luan et al., 2017), where S2S and AE are trained alternately, our approach trains S2S and AE at the same time by minimizing the loss function of Equation 3. 3.4 Inference Like Zhao et al. (2017); Bowman et al. (2016), for a given context, we sample different latent vectors to obtain multiple hypotheses. This is done by adding a random vector r that is uniformly sampled from a hypersphere of radius | r | to the prediction z S2S ( x ) . where | r | is tuned on the validation set to optimize the trade-off between relevance and diversity. z ( x, r ) is then fed to the decoder as the initial state of GRU cells. We then generate responses using greedy decoding. 4 4 Experiment Setup 4.1 Datasets We used the following datasets. Some of their key features are presented in Table 1. Switchboard: We use the version offered by Zhao et al. (2017), which is an extension of the original version by Godfrey and Holliman (1997). Zhao et al. (2017) collected multiple references for the test set using information retrieval (IR) techniques followed by human filtering, and randomly split the data into 2316/60/62 conversations for 4 Although we use greedy decoding in this work, other decoding techniques, such as beam search, can be applied. train/validate/test, respectively. Each conversation has multiple turns and thus multiple ( x, y ) pairs, as listed in Table 1. As our approach does not utilize extra information except conversation history, we removed the meta data (e.g. gender, age, prompt) from this dataset. Reddit: As the Switchboard dataset is relatively small and multiple references are synthetically constructed, we have developed another multi-reference dataset by extracting posts and comments on Reddit.com during 2011 collected by a third party. 5 As each Reddit post and comment may have multiple comments, it is a natural source of multi-reference responses. We further filtered the data based on the number of replies to obtain the final conversation dataset in which each context has at least 10 different responses, and on average the number of responses is 24.1 for a given context. The size is significantly larger than Switchboard, as listed in Table 1. The conversations are randomly shuffled before being split into train/valid/test subsets. Both encoders and the shared decoder consist of two GRU cells, each with 128 hidden units. The variance of the noise layer in each decoder is  2 = 0 . 1 2 . The word embedding dimension is 128. The weight parameters (see Equation 3) are set as  = 1 and  = 30 . For both datasets, the inference radius | r | (see Equation 4) is set to 1.5 which optimizes F1 score on the validation set. All models are trained using the Adam method (Kingma and Ba, 2014) with a learning rate of 0.001 on both datasets until convergence (around 4 epochs for Reddit and 10 epochs for Switchboard). For a given context x , we have N r reference responses and generate the same number of hypothe-5 ses. 6 We define the following metrics based on 4-gram BLEU (Papineni et al., 2002), as suggested by Zhao et al. (2017). Precision = 1 N r N r (cid:88) i =1 max j  [1 ,N r ] BLEU ( r j , h i ) Recall = 1 N r N r (cid:88) j =1 max i  [1 ,N r ] BLEU ( r j , h i ) F1 = 2 precision  recall precision + recall We use Precision as an approximate surrogate metric for relevance and Recall for diversity. It should be noted that recall is not equivalent to other diversity metrics, e.g., distinct (Li et al., 2016a) and entropy (Zhang et al., 2018), which only depend on hypotheses. One potential issue of these metrics is that even randomly generated responses may yield a high diversity score. F1 is the harmonic average of these two and is used to measure the overall response quality. We conduct a human evaluation using crowdwork-ers. For each hypothesis, given its context, we ask three annotators to individually measure the quality, on a scale of 1 to 5, in terms of two aspects: relevance and interest. Interestingness is treated as an estimation of the diversity, as these two are often correlated. The hypotheses from all systems are shuffled before being provided to annotators. System names are invisible to the annotators. S2S+Sampling: We consider a vanilla version of S2S model. The dimensions are similar to our model: both encoder and decoder consist of two stacked GRU cells with 128 hidden units, and the word embedding size is 128. As in the baseline in Zhao et al. (2017), we applied softmax sampling at inference time to generate multiple hypotheses. CVAE+BOW: For the CVAE conversation model, we use the original implementation and 6 We set the number of hypotheses equal to the number of references to encourage precision and recall have comparable impact on F1 hyperparameters of Zhao et al. (2017) with the bag-of-words (BOW) loss. The number of trainable model parameters is 15.4M, which is much larger than our model (3.2M). MTask: Since our approach utilizes a multi-task learning scheme, we also compare it against a vanilla multi-task learning model, MTask, similar to (Luan et al., 2017), to illustrate the effect of space fusion. The model architecture and hyperparameters are identical to the proposed model except that the loss function is L =  log p ( y | z S2S )  log p ( y | z AE ) . In this section, we undertake an in-depth analysis to verify whether the latent space induced by our method manifests desirable properties, namely: 1) disentangled space structure between relevance and diversity, 2) homogeneous space distribution in which semantics changes smoothly without holes. We first provide a qualitative investigation based on real examples. Then, we present a set of corpus-level quantitative analyses focused on geometric properties. In Table 2, we investigate three different directions from the context  Anyone want to start this game?  , which is a real example taken from Reddit. The three different directions correspond to clearly different semantics:  No I don't ,  when?  and  Yes I do.  If we generate a response with the vector predicted by the S2S model ( u = 0 ), our model outputs  I would love to play this game  which is highly relevant to the context. Now as we move along each direction, we can see our model gradually transforms the response toward the corresponding responses of each direction. For instance, towards  No I don't , our model gradually transforms the response to  I am not interested in the game  ( u = 0 . 18 ) and then  I am not interested.  ( u = 0 . 21 ). In contrary, towards  Yes I do , the response transforms to  I would love to play it.  ( u = 0 . 15 ). Besides the positive or negative directions, the same transition applies to other directions such as  When? . This example clearly shows that there is a rough correspondence context x : Anyone want to start this game? between geometric properties and semantic properties in the latent space induced by our method as shown in Figure 1 the relevance of the response decreases as we move away from the predicted response vector and different directions are associated with semantically different responses. In order to quantitatively verify the correspondence between direction and diversity , we visualize the distribution of cosine similarities among multiple references for each context for a set of 1000 random samples drawn from the test dataset. Specifically, for a context x k and its associated reference responses [ y k, 0 , y k, 1 ,    ] , we compute the cosine similarity between z AE ( y k,i )  z S2S ( x k ) and z AE ( y k,j )  z S2S ( x k ) . In Figure 3, we compare the distribution of our model with that of MTask, which does not employ our regularization terms. While our method yields a bell shaped curve with average cosine similarity being close to zero (0.38), the distribution of MTask is extremely skewed with average cosine similarity being close to 1 (0.95). This indicates that the directions of the reference responses are more evenly distributed in our latent space whereas everything is packed in a narrow band in the MTask's space. This essentially makes the inference process simple and robust in that one can choose arbitrary directions to generate diverse responses. alize the perplexity of reference responses along the path from the associated z S2S ( u = 0 ) to the z AE ( u = 1 ) corresponding to the predicted response. In Figure 4, we compare our model with MTask, which as already noted, does not employ our regularization terms. While our model shows a gradual increase in perplexity, there is a huge bump for MTask's line. This clearly indicates that there is a rough correspondence between distance and relevance in our latent space whereas even a slight change can lead to an irrelevant response in the MTask's space. We further illustrate the smooth change in relevance according to distance for a specific example in Table 3. Given the context  Anyone want to start this game? , our model   interp =  AE  interp =  S2S w/o regularization w. is able to transition from the predicted response  I would love to play this game  to a one of reference responses  Yes I do . The relevance smoothly descreases, generating intermediate responses such as  I would love to play it.  In contrary, the MTask model tends to produce irrelevant or ungrammatical responses as it moves away from the predicted response. Other desirable properties, with which we want to equip our latent space are homogeneity and convexity . If the space is not homogeneous, we have to sample differently depending on the regional traits. If the space is not convex, we have to worry about running into the holes that are not properly associated with valid semantic meanings. In order to verify homogeneity and convexity, we visualize our latent space in a 2D space produced by the multidimensional scaling (MDS) algorithm (Borg and Groenen, 2003), which approximately preserves pairwise distance. For comparison, we also provide a visualization for MTask. As shown in Figure 5, our latent space offers great homogeneity and convexity regardless of which model is used to produce a dot (i.e. z S 2 S or z AE ). In contrary, MTask's latent space forms two separate clusters for z S 2 S and z AE with a large gap in-between where no training samples were mapped to. We let each system generate 100 hypotheses { h j } for each context x i in the test dataset. Assuming x i has N r,i references, we pick the top N r,i distinct hypotheses ranked by log p ( h j | x i )+  | h j | . Similar to (Li et al., 2016a; Wu et al., 2016), we takes | h j | into consideration, as BLEU is sensitive to length. For fair comparison,  is tuned such that the average hypothesis length becomes roughly the same for all systems and approaches the average length of the references. 7 The automatic evaluation results are reported in Table 4. On both datasets, the proposed system consistently outperforms the baselines by a large margin in Precision, Recall, and F1. Examples of system outputs and human references can be found in Table 5 and Table 6 for Reddit and Switchboard, respectively. As shown in the examples, CVAE+BOW and other baseline models may generate diverse but not-so-relevant responses. We randomly sampled 500 contexts from the Reddit test dataset and picked the top 1 hypothesis generated for each context ranked by log p ( h j | x i ) +  | h j | . As in the automatic evaluation, we tuned  such that all systems have roughly 7 Approximately 10 words/tokens for Switchboard and 12 for Reddit context Everything about this movie is awesome! We also randomly select one reference for each context and compare them with the systems (labeled human in Table 7) As illustrated in Table 7, the proposed model outperforms all systems except human, consistent with our automatic evaluation results. We propose a SPACEFUSION model to jointly optimize diversity and relevance that leverages novel regularization terms to essentially fuse the latent space of a S2S model with that of an autoen-context coder model. This fused latent space exhibits desirable properties such as smooth semantic interpolation between two points. The distance and direction from the predicted response vector roughly match relevance and diversity, respectively. These properties also enable intuitive visualization of the latent space. Both automatic and human evaluation results demonstrate that the proposed approach brings significant improvement compared to strong baselines in terms of both diversity and relevance. In future work, we will provide theoretical justification of the effectiveness of the proposed regularization terms. We expect that this technique will find application as an efficient mixing board for conversation that draws on multiple sources of information.
abstain abstain objective result abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method objective abstain abstain method method abstain method result abstain result result abstain other abstain other other other other other other other other method other other other other other other other abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain other abstain abstain other abstain abstain abstain abstain abstain abstain method method abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain abstain abstain method method abstain abstain abstain method method method method abstain method abstain result abstain abstain abstain abstain result abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain objective result