Generating open-domain conversational responses in the desired style usually suffers from the lack of parallel data in the style. Meanwhile, using monolingual stylistic data to increase style intensity often leads to the expense of decreasing content relevance. In this paper, we propose to disentangle the content and style in latent space by diluting sentence-level information in style representations. Combining the desired style representation and a response content representation will then obtain a stylistic response. Our approach achieves a higher BERT-based style intensity score and comparable BLEU scores, compared with baselines. Human evaluation results show that our approach significantly improves style intensity and maintains content relevance. Linguistic style is an essential aspect of natural language interaction and provides particular ways of using language to engage with the audiences (Kab-bara and Cheung, 2016). In human-bot conversations, it is crucial to generate stylistic responses for increasing user engagement to conversational systems (Gan et al., 2017). Currently, most of the existing parallel datasets are not stylistically consistent. Samples in these datasets are usually contributed by a variety of users, resulting in an averaging effect across style characteristics (Zhang et al., 2018a). Meanwhile, constructing a parallel stylistic dataset for training the open-domain conversational agents is both labor-intensive and time-consuming. Recent studies show the effect of stylizing responses using a monolingual dataset in the desired style and a conventional conversational dataset (Niu and Bansal, 2018; Gao et al., 2019b). However, increasing style intensity often leads to  Corresponding author. the expense of decreasing content relevance between dialogue history and response. As an example in Figure 1 shows, Niu and Bansal (2018) independently train a response generation model and a stylistic language model and subsequently interpolates them in the inference phase. Lacking the interaction between the stylistic language model and response generation encoder, it usually yields a trade-off between style intensity and content relevance. Gao et al. (2019a,b) fuse a structured latent space where the direction denotes the diversity, and the distance denotes style intensity and content relevance. The main issue is that style intensity and content relevance are contradictory in measurement but are coupling to the same distance metric of the latent space. To sum up, the key issue of the above studies is the improper entanglement of style and content. To address the issue, we propose to disentangle the style and content of a response. The disentanglement is conducted on the structured latent space, where each sentence (dialogue history, response, and stylistic sentence) is projected into a vector representation. We further split the representation into two components: style and content representations. The former is a corpus-level feature since sentences within a dataset have the same style. In contrast, the content representation is a sentence-level feature decided by a sentence itself. We thus disentangle the content and style by diluting sentence-level information in the style representation. This encourages the encoding of content information into the content representation. Otherwise, the content information will be corrupted in the style representation, making it hard to reconstruct the original content in the subsequent decoding process. We conduct experiments on DailyDialogue conversational dataset (Li et al., 2017) and Holmes monolingual stylistic dataset (Gao et al., 2019b). Experimental results show that our proposed approach improves style intensity and maintains content relevance. Our contributions are listed below:  We propose a unified framework to simultaneously improve style intensity and maintain content relevance for neural stylistic response generation. We introduce a scheme of learning latent variables by a diluting strategy to disentangle the style and content. Experimental results show that our approach achieves higher performance in style intensity without decreasing content relevance, compared with previous approaches. The task of stylistic response generation is defined as follows: given a monolingual stylistic dataset S = { S 1 , ..., SN } 1 and a conversational dataset C = { ( X 1 , Y 1 ) , ..., ( XM , YM ) } , where S i , X i , and Y i denote a stylistic sentence, dialogue history, and a response respectively, the goal is to learn a generation model P (  Y | X ) , where  Y is a generated response expected to be in the style of S (called the desired style in the following sec-tions). We will first briefly review the concept of structured latent space and then introduce our disentanglement approach. Overview The structured latent space is constructed by two main mechanisms: (i) sharing a decoder between a sequence-to-sequence (S2S) model and an auto-encoder (AE), and (ii) fusion and smoothness objectives. As an example in Figure 2 shows, a response representation ZAE ( Y i ) is regularized by the two mechanisms to be distributed around its dialogue history representation Z S2S ( X i ) . The notations ZAE (  ) and Z S2S (  ) denote the representations computed by AE encoder and S2S encoder, respectively. Such a latent space makes it possible to predict a response  Y by sampling nearby the dialogue history representation. Based on that, Gao et al. (2019b) further align stylistic sentence representations into the latent space, which improves the style intensity of generated responses. In summary, the construction of the structred latent space is a process of aligning the three spaces ( Z S2S ( X i ) , ZAE ( Y i ) , and ZAE ( S j ) ) by two mechanisms (sharing the decoder, and fusion and smoothness objectives). Fusion Objective cross-aligns sentences of different spaces. Since X i and Y i are paired, we align them by minimizing their pair-wise dissimilarity: d conv = (cid:88) i  batch d E ( Z S2S ( X i ) , ZAE ( Y i )) n  l , (1) where d E denotes the Euclidean distance, n is the batch size, and l is the dimensionality of the latent space. cannot be applied to stylistic sentences since they are not paired with conversational data. To this end, the fusion objective instead optimizes the nearest neighbor distance between the two datasets: d style =12 d crossNN ( { Z S2S ( X i ) } , { ZAE ( S j ) } ) +12 d crossNN ( { ZAE ( S j ) } , { Z S2S ( X i ) } ) , (2) where d crossNN ( { a i } , { b j } ) denotes the batch average distance between a i and its nearest neighbor in the set { b j } . To further encourage the representations spread-out the latent space, a inner-distance loss is introduced: d spread-out = min { d innerNN ( Z S2S ( X i )) , d innerNN ( ZAE ( Y i )) , d innerNN ( ZAE ( S j )) } , (3) where d inner NN ( { a i } ) denotes the batch average distance between a i and its nearest neighbor in the set { a i } . The final fusion objective is defined as: L fuse = d conv + d style  d spread-out . (4) Smoothness Objective aims to make the structured latent space a continuous space, where each point can decode a natural sentence. Given three discrete points Z S2S ( X i ) , ZAE ( Y i ) , and ZAE ( S j ) , the objective encourages points in the area between Z S2S ( X i ) and ZAE ( Y i ) to generate Y i : Z conv = U Z S2S ( X i ) + (1  U ) ZAE ( Y i ) + (cid:15), L smooth,conv =  log P ( Y i | Z conv ) , (5) where (cid:15)  N (0 ,  2 I ) , and U  U (0 , 1) . Meanwhile, as a point moves from ZAE ( Y i ) to ZAE ( S j ) , the corresponding generation is expected to gradually move from Y i to S j : Z style = U ZAE ( Y i ) + (1  U ) ZAE ( S j ) + (cid:15) L smooth,style =  U log P ( Y i | Z style )  (1  U ) log P ( S j | Z style ) . (6) The smoothness objective L smooth is the sum of L smooth,conv and L smooth,style , and is added to the final loss function along with the fusion objective and response generation loss of S2S. Despite aligning monolingual stylistic sentences into the structured latent space helps stylize generated responses, their style intensity is still limited. We conjecture this is due to the coupling of the style and the content in sentence representations. To this end, we propose to disentangle the two aspects in the structured latent space. In our proposed approach, a sentence representation Z  R l in the latent space consists of two components: content representation Z c  R l c and style representation Z s  R l s , where l is the dimensionality of latent space and l c + l s = l . Z s encodes all the style information of a sentence. It is a corpus-level feature because Z s for different sentences in the same corpus should be similar. In contrast, Z c can be seen as a sentence-level feature which only decided by the content of its corresponding sentence. Figure 3 shows an example of our approach, where Z c and Z s can be seen as two contain-ers. Colored squares represent the content and style information. We encourage the disentanglement of the two types of information by diluting sentence-level content information in Z s . As an example in Figure 3 (a) shows, the content and style information may be mixed in both Z c and Z s . During the decoding process of a sentence, i.e., Y i , we replace its style representation Z s AE ( Y i ) with its batch average style representation  Z s AE ( Y i ) = 1 n (cid:80) j  batch Z s AE ( Y j ) . In this way, its sentence-level content information will be diluted since it greatly varies from other sentences' content information, which introduces extra noise. In contrast, its corpus-level style information, which is similar to that of other sentences within the batch, will remain unaffected. As the training processes, the content information will be encouraged to be encoded into Z c where it can remain unchanged, as an example in Figure 3 (b) shows. Otherwise, the content information will be corrupted in Z s , making it hard to recover the content of Y i . As a result, the encoding process will be punished by the response generation loss of S2S and the reconstruction loss of AE, as shown in Figure 3 (a). Based on that, we update the response generation process by replacing its style representation Z s with the corresponding batch average style representation  Z s : L S2S =  log P ( Y i | [ Z c S2S ( X i ) :  Z s S2S ( X i )]) , (7) where the bracket [:] denotes concatenation. The decoding process in the smoothness objective is updated similarly. Note that when we move from Content Information of Sentence #1 (S 1 ) Content Information of Sentence #2 (S 2 ) Style Information Content Representation Z c Style Representation ZSS 1 : Could you please tell me how I can go job-hunting in the web? The batch average style representation  Z s remains consistent with the target, i.e., being  Z s AE ( S j ) when the target is S j . The updated smoothness objective is as follows: L smooth,conv =  log P ( Y i | [ Z c conv :  Z s AE ( Y i )]) , L smooth,style =  U log P ( Y i | [ Z c style :  Z s AE ( Y i )])  (1  U ) log P ( S j | [ Z c style :  Z s AE ( S j )]) . (9) The final training loss is the sum of the response generation loss, fusion objective, and smoothness objective: L = L S2S + L fuse + L smooth . Here, we do not employ pre-training models, i.e., DialoGPT (Zhang et al., 2020b) and OpenAI GPT2 (Radford et al., 2019). This is because the disentanglement is usually conducted on a sentence representation. While most of the pre-training models depend on the attention mechanism, and there is no static global sentence representation during the decoding process. To generate a stylistic response  Y i given dialogue history X i during the inference process, we first obtain Z c S2S ( X i ) by S2S encoder and subsequently sample Z c (  Y i ) from the hypersphere of Z c S2S ( X i ) with a mannually tuned radius r . After that, we generate  Y i by concatenating Z c (  Y i ) and  Z s AE ( S j ) , which is the batch average style representation of randomly sampled stylistic sentences. Considering the discrepancy between training and inference that content and style representations in different corpora have never been concatenated for generation, we propose a soft combination approach to introduce the desired style by interpolating Z s S2S ( X i ) and  Z s AE ( S j ) : Z s soft = Z s S2S ( X i ) +    Z s AE ( S j ) , (11) where  is the weight of the desired style. After that,  Y i is generated by the decoder whose hidden state is set to [ Z c (  Y i ) : Z s soft ] . To further balance style intensity and content relevance, we also employ the re-ranking strategy following Gao et al. (2019b). It samples N y candidate responses and re-ranks them by: s r =   P S2S (  Y i | X i )+(1   )  P style (  Y i ) , (12) where P S2S (  Y i | X i ) is the generation probability under a S2S model measuring the relevance. P style (  Y i ) is the probability that  Y i has the desired style. It is a interpolation between the probabilities of a neural-based classifier and a n-gram classifier: P style (  Y i ) =   P neural (  Y i ) + (1   )  N (cid:88) n =1 w n  P n-gram (  Y i ) , (13) Training Dialogues 11,118 Validation Dialogues 1,000 Test Dialogues 1,000 Average Tokens Per Dialogue 114.7 Average Tokens Per Utterance 14.6 Table 1: Statistics of the DailyDialog dataset. where w n is a weight which is set to the accuracy of the corresponding classifier. Conversational Dataset We employ DailyDialog 2 (Li et al., 2017) as our conversational dataset C . It is a human-written multi-turn dataset covering various topics of daily life. Table 1 shows some statistics of its training, validation, and test set. We split dialogue of K utterances into K -1 samples. Each sample consists of at most three continuous utterances. The last utterance of a sample is regarded as the response. The previous utterances of the response are concatenated as its dialogue history. Here, Reddit dataset is not employed as Gao et al. (2019b) because the post-reply format data collected from social networks is noisy and different from real conversations (Li et al., 2017). Monolingual Stylistic Dataset Following Gao et al. (2019b), we use Holmes 3 as the stylistic dataset S . It is collected from the Sherlock Holmes novel series and consists of roughly 38k sentences. We do not use the arXiv dataset as it contains too many special tokens, i.e., equations, and incomplete sentences, such as is concerned and ex-actly identical restrictions. We compare the proposed approach with the following baselines: S2S , the sequence-to-sequence response generation model (Shang et al., 2015). S2S+LM , a S2S trained on C and a stylistic language model trained on S (Niu and Bansal, 2018). During the inference process, it generates a stylistic response by interpolating outputs of the two models. 2 http://yanran.li/dailydialog 3 https://github.com/golsun/StyleFusion Model Time (s) # of parameters S2S 4.55 63M Style Fusion 4.60 75M Ours 4.60 75M Table 2: The average running time (in seconds per batch) and the number of parameters. Style Fusion , a multi-task learning based model whose latent space fuses dialogue history, responses, and stylistic sentences with a specific structure (Gao et al., 2019b). Note that we do not consider the Label-Fine-Tuning model and Polite Reinforcement Learning model (Niu and Bansal, 2018), because they require some training samples in the conversational dataset to have the desired style (Gao et al., 2019b). We implement the proposed approach based on the released code of Style Fusion model 4 . The vocabulary table consists of the most frequent 20,000 words. S2S encoder, AE encoder, and the shared decoder are two-layer LSTMs. The number of their hidden units is 1000, which is also the size of the structured latent space. The dimension of Z c and Z s is 950 and 50, respectively. The maximum length is set to 90 for the dialogue history and 30 for the response. During the training process, we use the ADAM optimizer, whose learning rate is 0.0003. 2 for sampling (cid:15) in Equation 8 is 0 . 1 2 . Table 2 shows the average running time on a single TITAN X (Pascal) GPU. During the inference process, the weights  and  for re-ranking are set to 0.5. The weight (accuracy) of n-gram classifier is 0.93, 0.87, 0.77, and 0.65 for n from 1 to 4. The number of candidate responses, N y , is set to 10. The radius r is set to 3. Automatic Evaluation Considering that it is unfair to evaluate a response by the classifiers that are used for selecting the response (Song et al., 2020), we fine-tune a BERT (Devlin et al., 2019) to measure style intensity. Concretely, positive samples are the stylistic sentences. Negative samples are 4 https://github.com/golsun/StyleFusion Model SI(%) Dist-1 Dist-2 BLEU-3 BLEU-4 Mean S2S (Shang et al., 2015) 6.32 0.035 0.227 0.70 0.20 0.10 S2S+LM (Niu and Bansal, 2018) 32.79 0.015 0.086 0.55 0.08 0.13 Style Fusion (Gao et al., 2019b) 10.58 0.043 0.280 0.82 0.22 0.14 Ours (  =0.25) 11.91 0.041 0.275 0.79 0.23 0.16 Ours (  =0.50) 20.67 0.040 0.275 0.64 0.17 0.19 Ours (  =0.75) 34.85 0.038 0.285 0.47 0.10 0.16 Table 3: Automatic evaluation results of SI, Dist-1, Dist-2, and BLEU. randomly selected from DailyDialog's responses, which are of the same amount of sentences as the positive samples. Given the fine-tuned BERT classifier (whose accuracy achieves 0.96 on the validation set), we report the average probability of responses being positive as a measurement of the style intensity. For brevity, we denote this metric as SI . The content relevance is evaluated by BLEU. Since it may correlate weakly with human judgments of quality in a single reference setting (Liu et al., 2016), we employ the expanded responses in multi-reference DailyDialog test set (Gupta et al., 2019) as references to alleviate the problem. Meanwhile, we evaluate the diversity by Dist-k (Li et al., 2016), which is the number of distinct k -grams normalized by the total number of words of responses. Human Evaluation We randomly sample 200 messages from the test set of C to conduct the human evaluation from two aspects: style intensity and content relevance. Each aspect is independently evaluated by five Amazon Mechanical Turk (AMT) 5 workers whose approval rate is greater than 95%, and the number of approved is greater than 500. Given dialogue history and two responses generated by a baseline and our approach, the workers are asked to give a preference of which one is 5 https://www.mturk.com Content Relevance Style Intensity Win Lose Win Lose vs. S2S 40.21 39.79 49.37 36.84 vs. S2S+LM 65.00 20.00 53.30 32.50 vs. Style Fusion 43.32 42.67 48.77 36.68 Table 4: Pair-wise human evaluation results of content relevance and style intensity. Figure 4 shows the trade-off between style intensity and content relevance in our approach. There is an improvement in SI and a decrease in BLEU associated with the increase of  in Equation 11. To assess the overall performance, we also compute their harmonic mean, whose maximum lies around  = 0 . 5 . We thus conduct the human evaluation and analysis in this parameter setting. We report the human evaluation results in Table 4. Our approach is clearly preferred in style intensity because the percentage of Win is significantly higher than that of Lose ( p < 0.001, T-test). In terms of content relevance, the ratios of Win in vs. S2S and vs. Style Fusion are similar to those of Lose. This suggests that our approach can significantly improve the style intensity without decreasing the content relevance. In contrast, S2S+LM loses in most of the cases in the content relevance. Following Zhou et al. (2018) and Ke et al. (2018), we evaluate the agreement of annotators via inter-rater consistency. The percentage of samples that at least three annotators have the same preference (3/5 agreement) is 81.80%. And the percentage for 4/5 agreement is 32.15%. Table 3 shows the results of the automatic evaluation. Our approach has the highest mean score, which indicates that it achieves the best overall performance. S2S+LM has a high SI score, but its BLEU scores are not as good as others, i.e., S2S. This is in line with our human evaluation results and Niu and Bansal (2018)'s observation that biasing a decoder with a stylistic language model may harm the content relevance. In contrast, our approach (  = 0 . 25 ) significantly outperforms S2S and is comparable to Style Fusion. By increasing  to 0.5, the BLEU score drops slightly but is comparable to baselines (evidenced by the human evaluation results). Meanwhile, there is a significant improvement (up to 95.37%) in SI comparing with Style Fusion. This verifies the effectiveness of our disentanglement approach in improving the style intensity and maintaining the content relevance. Besides, the Dist-k results in Table 3 also indicate that the diversity of our approach is comparable to the best-performed Style Fusion. We conduct ablation studies to investigate the contributions of the fusion objective, smoothness objective, and our disentanglement approach. To focus on their effects on the generation process, in this section, we sample a single response without using the re-ranking strategy (Equation 12). Table 5 shows the results of the ablation study. There is a significant decline in SI and a slight change in BLEU-3 and BLEU-4 after removing each component. This indicates that a multi-task learning architecture without the three components [ Z c : Z s ] Z s Style Fusion 0.83 0.72 (-13.02%) Ours 0.88 0.86 (-1.71%) Table 6: Style classification accuracy of the full latent variable ( [ Z c : Z s ] ) and Z s . can achieve a good content relevance performance but fails to stylize a response. By removing the disentanglement component, our approach degenerates into Style Fusion. In this case, the SI score decreases significantly while BLEU scores are nearly unchanged, which demonstrates the disentanglement could improve the style intensity and maintain the relevance at the same time. The decreases in SI after removing the fusion objective and smoothness objective are more significant than that after removing the disentanglement. This is because the two objectives are bottom components for constructing the structured latent space, where our approach and Style Fusion are built upon. In this section, we analyze whether style information is disentangled into Z s . To achieve this goal, we train style classifiers taking as input a latent variable and use the validation accuracy as an indicator. Taking our approach as an instance, we first freeze the parameters of our well-trained model. Then we independently learn two style classifiers whose inputs are the full latent variable ( [ Z c : Z s ] ) and Z s respectively. Note that Z c and Z s in Style Fusion are a simple partition of its latent variable. There are not any disentanglement approaches applied to obtain the two representations. As shown in Table 6, Style Fusion achieves 0.83 validation accuracy training on its full latent variable. And the accuracy decreases by 13.02% when the classification is only based on Z s . In contrast, the decrease of our approach is only 1.71%, indicating that most of the style information is disentangled into Z s . We show a visualization of the disentanglement of the latent variable by MDS (Borg and Groe-nen, 2005) in Figure 5. Each figure consists of Z s (black) and three continuous sub-sequences extracted from the head (yellow), middle (red), and tail (blue) of Z c . The sub-sequences are of the same length with Z s . For both stylistic and conversational samples, all the sub-sequences and Z s are mixed in Style Fusion. In contrast, there is a clear separation between Z s and the sub-sequences Dialogue  Yes , after my graduation, History I worked in a trade company in Macao for one year. in our approach. This is because most of the style information is disentangled into Z s in our approach, making its distribution different from sub-sequences of Z c . Table 7 shows some examples of generated responses. There is no significant Holmes style in responses of S2S. Similarly, the style intensity of responses in Style Fusion is also limited. The semantics of S2S+LM's response in the first example is not very clear, making it less relevant to the dialogue history than other responses. We believe this is also due to the lack of interaction between the response generation encoder and the stylistic language model. In contrast, our approach not only achieves a good content relevance performance but also has a significant Holmes style, which is quite polite and formal. The task of text style transfer aims at transferring the style of a sentence while preserving its meaning. One way is to disentangle the content and style, and subsequently combine the content with the desired style. The disentanglement can be achieved by adversarial learning (Shen et al., 2017; Hu et al., 2017; Fu et al., 2018; Yang et al., 2018; Lo-geswaran et al., 2018), reinforcement learning (Jain et al., 2019), back-translation (Prabhumoye et al., 2018; Nogueira dos Santos et al., 2018), multi-task learning (John et al., 2019), and removing stylistic phrases (Li et al., 2018; Xu et al., 2018; Zhang et al., 2018b). The other way transfers the style without disentangled representations, for example using generator-evaluator architecture (Gong et al., 2019), cycle reconstruction (Dai et al., 2019), parameter sharing (Wang et al., 2020), and data augmentation (Zhang et al., 2020a). The main difference between our task and text style transfer lies in two aspects. First, all the content to be generated is available in the input in text style transfer, while our task needs to create new (response) content. And the key is content relevance to the dialogue history, rather than content preservation of the input. Second, the data for text style transfer is isomorphic. Data in different styles are in the same free-text format. However, our conversational data are context-response pairs while the stylistic data are free-texts, which is heterogeneous and requires more sophisticated structures, i.e., the structured latent space (Gao et al., 2019b). Niu and Bansal(2018) propose three weak-supervised models based on reinforcement learning, conditional text generation, and language model. Gao et al. (2019b) fuses the latent spaces of a response generation model and a stylistic auto-encoder to improve the style intensity of sampled responses. Yang et al. (2020) inject the style information by introducing a word-level KL loss and a sentence-level style classifier to the fine-turning process of DialoGPT (Zhang et al., 2020b). Distinct from previous work, we explicitly disentangle the style and content in the latent space and employ a unified architecture to jointly optimize the style intensity and content relevance. We propose a uniform framework to simultaneously improve the style intensity and maintain the content relevance for neural stylistic response generation. In contrast to existing approaches, our approach disentangles the style and the content in the latent space by a diluting strategy. Experiments show that our approach improves the style intensity of generated responses and maintains the content relevance at the same time, which demonstrates the effectiveness of this approach. The authors would like to thank all the anonymous reviewers for their insightful comments. The authors from HIT are supported by the National Natural Science Foundation of China (No. 62076081, No. 61772153, and No. 61936010) and Science and Technology Innovation 2030 Major Project of China (No. 2020AAA0108605). The author from UCSB is not supported by any of the projects above. This paper honors the ACL Code of Ethics. Stylistic response generation intends to improve the engagement of a dialogue system in human-bot conversations. It responds to users with the desired style, i.e., being polite, humorous, or romantic, rather than imitating any specific person. Meanwhile, style is a linguistic aspect of natural language interaction. There is not any identity characteristic being used as a variable.
abstain abstain objective abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain method abstain abstain abstain abstain abstain method objective objective abstain result other method other other other other other other other other other method other other other other other other other other abstain method method method abstain abstain abstain method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain method abstain other abstain method abstain abstain method abstain method method abstain method abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain other abstain abstain abstain abstain method method method abstain other result abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain result abstain result abstain abstain abstain result result abstain method abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain other other other other abstain objective other other other abstain abstain abstain abstain method objective objective objective other other other abstain abstain abstain abstain abstain