Towards building intelligent dialogue agents, there has been a growing interest in introducing explicit personas in generation models. However, with limited persona-based dialogue data at hand, it may be difficult to train a dialogue generation model well. We point out that the data challenges of this generation task lie in two aspects: first, it is expensive to scale up current persona-based dialogue datasets; second, each data sample in this task is more complex to learn with than conventional dialogue data. To alleviate the above data issues, we propose a data manipulation method, which is model-agnostic to be packed with any persona-based dialogue generation model to improve its performance. The original training samples will first be distilled and thus expected to be fitted more easily. Next, we show various effective ways that can diversify such easier distilled data. A given base model will then be trained via the constructed data curricula, i.e. first on augmented distilled samples and then on original ones. Experiments illustrate the superiority of our method with two strong base dialogue models (Transformer encoder-decoder and GPT2). The ability to generate responses with consistent personas is important towards building intelligent dialogue agents. In past years, there has been a growing interest in introducing explicit personas in dialogue generation models (Song et al., 2019; Wolf et al., 2019). A piece of persona text generally consists of profiles and background personal facts. A clipped persona-based dialogue from the PersonaChat (Zhang et al., 2018a) dataset is shown in Figure 1, which covers rich persona features. For  Work was done when Yu Cao was an intern at Tencent AI Lab. a persona-based dialogue generation model, generated responses need to be relevant to the dialogue context as well as consistent with personas. Most existing generation models for this task rely heavily on training with sufficient persona-based dialogues. However, available data are limited due to their expensive collection costs. Take the PersonaChat as an example, two crowd-sourced annotators are hired to play the part of a provided persona and converse naturally with each other. In total, about 162 thousand dialogue utterances are collected with less than 5 thousand unique persona profiles. Compared with conventional dialogue datasets such as OpenSubtitles (Lison and Tiedemann, 2016) and Weibo (Shang et al., 2015) with millions of utterances, persona-based dialogue datasets are relatively small. Besides the limited data scale, another data issue we want to point out is that a persona-based dialogue is more complex to learn with, in comparison with conventional dialogues. Recall that a persona-based dialogue involves not only multiple dialogue utterances, but also auxiliary persona sentences. Welleck et al. (2019) showed that not all responses in the PersonaChat dataset are consistent with the provided personas. This makes it difficult for a model to capture a reliable mapping from training data. Supposing we apply a similar dialogue model as in conventional dialogue generation tasks with a comparable parameter size, we 7984 should expect more data would be necessary to train a robust model on the more difficult data setting. Moreover, it may be difficult to use existing data augmentation methods (Li et al., 2019; Niu and Bansal, 2019) to automatically construct such complex persona-based dialogue data. For example, if we apply back translation (Sennrich et al., 2016) to every sentence in persona-based samples, the augmented ones may not maintain the coherence between the dialogue history and the response as well as the consistency between the persona and the response simultaneously. A few studies have been conducted to alleviate the above data issues by finetuning existing pretrained models such as GPT (Wolf et al., 2019; Golovanov et al.) or BERT ( ? Song et al., 2021). They often stick to a certain pretrained model. Sophisticated finetuning strategies, including proper network modifications and loss functions, are required to get satisfactory performance, making them not useful across different pretrained models. Moreover, they do not address the data difficulty issue explicitly. Most of them simply concatenate all persona and dialogue history sentences into a single input sequence for finetuning, and rely on the ability of the pretrained model to fast adapt to the target data domain. Hence, we want to design a model-agnostic method to address both the data scale and data difficulty issue, which can be packed with any base model, either trained from scratch or finetuned from a pretrained model. In this work, we propose a data manipulation method for persona-based dialogue data, which is model-agnostic to be packed with any base model to improve their robustness and consistency. Our method includes three operations on data, namely D 3 , in sequence: (i) D ata distillation: original training samples are simplified into contain only useful and less redundant persona sentences and dialogue utterances, which are expected to be fitted more easily; (ii) D ata diversification: with the easier distilled samples, we can also perform data augmentation more reliably. We design various methods to edit new personas, and then align them with new and consistent responses to improve data diversity; (iii) D ata curriculum: with both augmented distilled and original data at hand, we arrange them into a data curriculum for model learning (Bengio et al., 2009), where the base model is trained on the easier augmented distilled data and then the harder original data. To validate the effectiveness of our method, we perform experiments on two strong base dialogue models, Transformer-based encoder-decoder and GPT2. Persona-based dialogue generation It sees growing interest in recent years, thanks to the released benchmark datasets such as PersonaChat/ ConvAI2 (Zhang et al., 2018a; Dinan et al., 2020). Previous works mostly focus on modifying dialogue models to condition auxiliary persona information, including extra persona embedding(Li et al., 2016b), profile memory (Zhang et al., 2018a), copying from personas (Yavuz et al., 2019), CVAE with persona information (Song et al., 2019), and using meta-learning to augment low-resource personas (Tian et al., 2021). Recent works try to adopt large-scale pretrained models on this task. GPT/GPT2 (Radford et al., 2018, 2019) are chosen the most often and shown to improve the generation quality with different finetuning strategies (Wolf et al., 2019; Golovanov et al.; Cao et al., 2020). Some leverage BERT (De-vlin et al., 2019) as backbones ( ? Song et al., 2021). Other pretrained models also demonstrate their effectiveness (Lin et al., 2021). The aforementioned methods often need proper network modifications and finetuning loss functions in order to get satisfactory performance. It is hard to transfer them to be useful across different pretrained models. Moreover, most of them simply concatenate persona texts and dialogue history together as a single input sequence (Wolf et al., 2019; Roller et al., 2021), highly depending on the ability of the pretrained model to fast adapt to the target data domain. Text data manipulation Various data augmentation methods have been widely used in many NLP tasks (Sennrich et al., 2016; Hou et al., 2018; Guo et al., 2019; Min et al., 2020), which are also effective to boost the performance of dialogue models. New generated dialogue utterances (Li et al., 2019; Niu and Bansal, 2019) and retrieval results (Zhang et al., 2020) can be used to augment the training data. However, all previous work only studies the pairwise relationship between a query and a response to design the augmentation techniques, which are not applicable to involving auxiliary information, such as personas, simultaneously. uninformative or noisy samples to enhance data quality (Csky et al., 2019; Akama et al., 2020). Cai et al. (2020a) combine data augmentation and re-weighting to make models learn more effectively. Tian et al. (2019) utilize learnable memory based on dialogue clusters to enhance the model. Curriculum learning Bengio et al. (2009) examine the benefits of training models using various curricula successively from easy to hard. It has been applied to many NLP tasks such as machine translation (Platanios et al., 2019), reading comprehension (Tay et al., 2019) and language understanding (Xu et al., 2020). Cai et al. (2020b) adopt the idea in open-domain dialogue generation, where curriculum plausibility is determined by the response properties, including coherence and diversity. Our work is different in that we introduce new distilled data regarding as a curriculum. We first formally define a persona-based training sample. It consists of L persona description sentences P = { p 1 , p 2 , .., p L } , M dialogue history utterances H = { h 1 , h 2 , .., h M } , and a gold response R . The given training dataset is denoted as D = { ( P, H, R ) } . Note that L and M in different training samples can be different. A dialogue model needs to generate a response  R , which is coherent with the dialogue history H and consistent with persona information in P . Our proposed data manipulation method D 3 is model-agnostic. For any dialogue model, we will not change the model itself, but only manipulate its training data. We develop three data manipulation operations in sequel, former two for augmentation and the last one eases training, shown in Figure 2: 1. D ata distillation. We construct simple persona-consistent data D dis = { ( (cid:101) P , (cid:101) H, (cid:101) R ) } by removing redundant information in P and H ; 2. D ata diversification. Due to the limited amount of distilled samples, we design various methods to increase the data variety and scale, and obtain the diversified data D div = { ( (cid:101) p, (cid:101) h, (cid:101) r ) } ; 3. D ata curriculum. We combine D dis and D div as the augmented dataset D a . A curriculum strategy is defined to train the model with the easier distilled samples in D a first and then the original ones in D . Before introducing our distillation method, we discuss the difficulty of training a model with the original original training samples in detail. The dependency of a response on the given persona fluctuates between different parts of the persona sentences. As shown in Figure 1, most responses only correspond to one persona sentence. The remaining persona information is mostly redundant, and may confuse the model to attend on useful persona information. Similarly, we notice that models tend to attend more on the last few utterances of H rather than the historical ones. We find that by using a Transformer encoder-decoder model, the attention weights of the last Transformer layer on the last utterance is 45% higher than the average on the other utterances. See Appendix C.1 for the experiment and results. This observation is also consistent with previous studies on multi-turn context understanding (Khan-delwal et al., 2018; Sankar et al., 2019). A few previous works have demonstrated that attention-based models will be distracted by noisy attended information, and accurate attention supervisions can be very beneficial (Liu et al., 2016; Hsu et al., 2018). Inspired by them, we mimic a hard attention supervision between the response and useful persona/dialogue history by directly removing redundant tokens in the attended sequences. Therefore, different from previous work that modify the model to inject attention supervisions, our method only manipulates data. Persona distillation We aim to determine which persona sentence the current response is consistent with, and thus remove the remaining non-consistent ones. To do so, we associate each persona sentence p k with the target response R , and determine the consistency between each p k and R . Following previous work (Welleck et al., 2019), we cast it as a natural language inference (NLI) problem. If R entails p k , it is considered to be consistent with p k , otherwise irrelevant to p k . A trained RoBERTa (Liu et al., 2019) model is used here as the NLI model, with an accuracy of 90.8% on the DialogueNLI dev set provided in Welleck et al. (2019). Details are provided in Appendix A.1. Dialogue history distillation We can adopt a trained attention-based model to determine useful context sentences. For simplicity, we could also keep only the most useful last utterance HM in a distilled sample (as suggested by our preliminary experiments discussed in the beginning of this sec-tion). In our experiments in 4, we find that using the last utterance is enough for our method to work 7986 P k Persona distillation ... A distilled sample ( (cid:101) P , (cid:101) H, (cid:101) R ) is ready to be constructed now. Here, (cid:101) P and (cid:101) H both contain only one sentence. (cid:101) P is any p k that entails R , and (cid:101) H is the last utterance in the dialogue history, and (cid:101) R = R . Such samples form the distilled dataset D dis . Note that an original sample in D may result in none, one, or multiple distilled samples, as R may entail none, one, or multiple persona sentences. Distilled samples should ease model training as their responses are highly dependent on their (cid:101) P and (cid:101) H . However, samples in D dis are limited in terms of both scale (around 40% of the original data) and diversity (about 4.5k unique persona sentences). Hence, it is necessary to augment D dis . Thanks to the assured relationship between (cid:101) P / (cid:101) H and R , we can devise possible methods to diversify distilled samples with more semantically varied samples. Our data diversification operation contains the following three parts along with quality filtering, as shown in Figure 2. Persona editing We aim to obtain new persona sentences to improve the data scale, and more importantly the persona diversity. Hence, we here consider both token-level and phrase-level editing methods given a persona sentence (cid:101) P :  Token-level editing: we randomly mask a pre-defined ratio of tokens in (cid:101) P , then use a pretrained BERT (Devlin et al., 2019) model to make predictions on the masked positions one by one. Phrase-level editing: we remove the last few tokens in (cid:101) P with the removal length determined by a random ratio, and utilize a pretrained GPT2 (Rad-ford et al., 2019) to rewrite the removal part. Multiple edited persona sentences can be obtained from one certain (cid:101) P . Here, we finetune pretrained models using all persona sentences for a trade-off between semantic diversity and domain similarity. To ensure a satisfactory fluency and novelty of an edited persona (cid:101) p , we rate it via a scoring function: f =   PPL ( (cid:101) p ) + (1   )  BS f ( (cid:101) p, (cid:101) P ) . Here, PPL calculates the normalized perplexity via a GPT2 model to measure its fluency, and the rescaled F1 value of BERTScore (BS f ) (Zhang et al., 2019) is employed to evaluate the semantic similarity between two sentences. Lower values for both functions are preferred, indicating higher fluency or novelty. is a hyper-parameter. We rank all edited personas originated from the same (cid:101) P with the ascending order of their scores in Eq. 1, and select the top N p ones. Response aligning Since the semantic meaning of an edited persona sentence obtained above could change, the original response may not be consistent with it. Therefore, we need to get a new aligned response to maintain the persona consistency. Two approaches are utilized to obtain an aligned response (cid:101) r given an edited persona sentence (cid:101) p and the corresponding distilled history utterance (cid:101) H :  Token-level editing: We observe that some overlapped tokens can be found between (cid:101) P and (cid:101) R . If an overlapped token w has been changed to a new token w (cid:48) in the edited persona (cid:101) p , we directly replace w in (cid:101) R with w (cid:48) in the same positions, resulting in an aligned response (cid:101) r . An illustration figure can be found in Appendix A.2. Model predicting: If no overlapped token can be found, token-level editing will not be applicable. Then we employ a GPT2-based encoder-decoder model (Cao et al., 2020) finetuned on the distilled 7987  P  p r  my husband is a lawyer. Dialogue history augmentation To further scale up the size of distilled samples, we also manipulate the dialogue history (cid:101) H . Since the diversity scarcity issue is not severe in (cid:101) H , we use a popular sentence-level data augmentation method, back translation (BT) (Sennrich et al., 2016), to obtain variants of dialogue utterances. We could consider the semantics of the variants are identical. Distilled history utterance (cid:101) H is translated into an intermediate language, then back into the source language using a couple of existing translation models. The original dialogue history and its N h variants compose the augmented dialogue history set { (cid:101) h } . Combining the above three parts together, we now obtain new samples { ( (cid:101) p, (cid:101) h, (cid:101) r ) } . We evaluate them with respect to fluency, persona consistency and history coherence: s =   PPL ( (cid:101) r ) +   NLI ( (cid:101) p, (cid:101) r ) + (1     ) NLI c ( (cid:101) h, (cid:101) r ) , (2) where NLI measures the entailment between a persona sentence and the response by the same NLI model in 3.1, and NLI c evaluates the entailment between a dialogue history utterance and the response using another NLI model (Dziri et al., 2019)(details in Appendix A.2). and  are hyper-parameters. We filter samples below a threshold T , and the remaining samples constitute the diversified data set D div . The whole augmented training dataset is the union of D dis and D div . The quality of augmented samples is discussed in Appendix B. D D dis D div D a D + D a #sample 65,719 26,693 26,700 53,393 119,112 #persona 4,710 4,522 9,788 14,310 14,498 #token 20,467 13,420 12,794 17,835 23,269 Table 1: Statistics of samples obtained in each stage. During inference, the model should be capable to handle testing data with multiple persona sentences and dialogue history utterances as the original data. Therefore, a model trained using D a only is not proper. We should use both D a and D . Unlike previous studies that treat the original and augmented data equally and mix them directly, we design a curriculum strategy. Considering the different training difficulty of data in D a and D , we treat D a as an easy curriculum while the original dataset D as a hard curriculum. The model is trained on such data curriculum successively until convergence. To validate the effectiveness of our proposed model-agnostic data manipulation method, we first experiment on two strong persona-based dialogue generation models (Transformer encoder-decoder and GPT2) on the benchmark PersonaChat (Zhang et al., 2018a) dataset. Next we conduct a series of analysis to examine the usefulness of different data manipulation operations in our method. 1 4.1 Experimental Setup Dataset The PersonaChat (Zhang et al., 2018a) data is widely used in this field (Song et al., 2019, 2020; Wolf et al., 2019; Golovanov et al.). Each sample has a dialogue history H with no more than 15 utterances ( M  15 ) and a persona P with between 4 and 6 sentences ( 4  L  6 ). Numbers of samples, unique persona sentences, and tokens in each stage of our method are listed in Table 1. Base models Two dialogue model architectures are considered:  TRANSFORMER (Vaswani et al., 2017): an encoder-decoder architecture using Transformer as the backbone with pointer generator (See et al., 2017) integrated;  GPT2: one of the most powerful pretrained models on this task (Wolf et al., 2019; Golovanov et al.; Cao et al., 2020). 1 Code is available at https://github.com/ caoyu-noob/D3 . TRANSFORMER is trained from scratch, and GPT2 is finetuned. For both models, we construct training data by concatenating persona and dialogue history as a single input sequence, in which special symbols and token type embeddings are involved to distinguish between them. The negative log-likelihood loss is used to train models using Adam optimizer (Kingma and Ba, 2015). Compared methods We pack two base models with our method D 3 and other data manipulation approaches for comparison:  BACKTRANSLATION (BT) (Sennrich et al., 2016): we perform BT on all sentences in a training sample, including the persona sentences and dialogue utterances, and train the model with the augmented and original data jointly;  CVAE (Li et al., 2019): a CVAE-based generation model is trained on the original data and then used to generate new responses via sampling with different latent codes. Since it can only handle pairwise data, we concatenate all input sentences as a single input sequence in this method;  ENTROPYFILTER ( FILTER ) (Csky et al., 2019): it removes generic responses according to the entropy, which is calculated using the dialogue history and the response without using the persona. The detailed configurations of each method are given in Appendix B. Automatic metrics We adopt multiple widely used metrics to measure the response quality, including Perplexity (PPL), BLEU (Papineni et al., 2002), NIST-4 (Doddington, 2002) and BERTScore (Zhang et al., 2019). We use the same BS f in Eq. 1 for BERTScore. To evaluate the response diversity, we use Distinct-n (Li et al., 2016a) (Dist, n=1,2,3) which is the ratio of unique n-grams among the corpus, and Entropy-n (Zhang et al., 2018b) (Ent, n=1,2,3) that is the entropy obtained via the n-gram distribution in a sentence. Moreover, C-score (Madotto et al., 2019) ( C ) is involved, where we follow the default setting and use the output of an NLI model trained on the DialogueNLI dataset (Welleck et al., 2019) to indicate the consistency between a response and persona sentences. Human evaluation We randomly selected 200 samples from the test set for human evaluations. Five professional annotators from a third-party company were asked to rate the responses from three aspects: 1) Fluency (Flu.); 2) Coherence (Coh.) with the dialogue history, 3) Persona consistency (Pcon.). The scores for the first two aspects have three scales, in which 1/2/3 indicates unac-ceptable/moderate/satisfactory respectively. The last one is binary, where 1 means the response is consistent with at least one persona sentence in the sample and 0 otherwise. The agreement rate from raters is 97.5%, 89.5%, 100% @3 (at least 3 of them reach an agreement) in the these aspects, indicating the validity of scores. The instruction of human evaluation is given in Appendix B. 4.2 Results Table 2 reports the results on two based models trained with the use of various compared data manipulation methods. T-test is conducted between our D 3 and other compared methods on each base model for metrics including BS f , C-score and three human evaluation metrics. Other automatic metrics have similar results or are not applicable such as Distinct-n. Details of the significant tests are given in Appendix C.2. On TRANSFORMER , all methods achieve improvements on most metrics compared with training with the original dataset. Our method yields the best performance except for Ent-1. On GPT2, many methods fail to improve the various metrics consistently. For example, on the persona consistency (Pcon.), only ENTROPY FILTER and our method can get higher scores than training with the original dataset. The reason is that the data scarcity issue is less severe with a pretrained model, and it is more important to address the data diversity issue. In our method, the augmented distilled samples are encouraged to have different semantics with the original ones and improve the data diversity, and thus continue to get improvements on the strong pretrained GPT2. We further analyze the contributions made by different data manipulation operations in our method by answering the following three questions: 1. Is there a need to construct simple data D dis as in data distillation? 2. Can data diversification effectively obtain diverse distilled data? 3. Does the curriculum strategy better exploit the augmented data and help model training? We use results on TRANSFORMER here for discussion in the following part. Refer to Appendix C.3 for extensive results on GPT2 model. Analysis of data distillation To examine the effectiveness of data distillation, we need to neutralize the influence of data diversification as it is only applicable to distilled data. Following variants of our D 3 are considered: 1) w/o diversification : only using distilled data D dis in the easy curriculum; 2) w/o distillation : based on 1), we recover samples in D dis into their original format, which means all their persona sentences and history utterances are included; 3) only distillation : only D dis is used in training without using the original data in D . Results of these variants are shown in the middle of Table 3. Obviously, removing data diversification decreases the performance in all aspects as the model has less training data. If we further remove data distillation and use the same amount of data in their original formats, the model performs even worse, especially on the C-score. This validates the effectiveness of data distillation in our method. However, it is not proper to completely rely on distilled data. From the results of only using distilled data in training, our method improves the C-score, yet significantly degenerates in other aspects. The reason is that the relationship between persona/dialogue history and the response has changed from the original data to their distilled ones. Thus a model trained with distilled data should serve as a warm start to learn the original data, but not to replace the original data. We also test the robustness of our data distillation method by using an NLI model trained in a few-shot setting (200 samples). Results are included in Table 3 as D 3 *. It is slightly worse than our method with sufficient NLI training data, but still superior to most compared methods. Note that the response diversity metrics nearly remain unchanged. This means that our data diversification methods are still effective when starting from noisy distilled samples. It also shows that our method can be useful when only limited in-domain NLI labeled data are available for data distillation. 1 shows that the diversified data contain many new persona sentences as well as tokens. Besides, we compute 7990 PPL BLEU NIST-4 BS f Ent-1 Ent-2 Ent-3 Dis-1 Dis-2 Dis-3 C TRANSD 3 37.30 3.358 1.206 0.1574 4.223 6.165 7.298 1.826 7.923 14.42 0.485 Original 38.28 3.140 1.148 0.1486 4.046 5.484 6.262 1.609 6.298 11.71 0.235 Only augment 126.3 1.603 0.956 0.0852 4.315 6.309 7.426 1.747 7.530 12.66 0.942 Shuffle 37.66 3.203 1.175 0.1521 4.128 6.096 6.979 1.659 6.889 13.79 0.404 Reverse 48.17 2.137 1.019 0.1508 3.947 5.291 6.039 1.368 5.503 9.211 0.912 Table 4: Performance comparison between different curriculum variants, using TRANSFORMER as the base model. the Novelty metrics (Wang and Wan, 2018; Zhang et al., 2020) of diversified samples in D div . It takes the original distilled samples in D dis as references, and uses the Jaccard similarity function to measure the proportion of n-grams ( n = 1 , 2 , 3 , 4 ) in D div but not in D dis . A higher value means more novel content. Note that we particularly prefer more novel personas, while not encouraging more novel dialogue histories. Thus, the Novelty scores on the overall samples which include dialogue histories, personas and responses, are lower than those on the personas. To further examine how each part of data diversification works, we conduct the following ablation studies: 1) w/o persona editing : no persona sentence will be edited; 2) w/o history augmentation : only original dialogue history is used; 3) w/o response filtering : all constructed samples are directly used without using Eq. 2. Results in the bottom of Table 3 show that all these designs contribute to the performance of the whole method. Among them, response filtering is the most important as it ensures the quality of augmented samples. We also investigate the proportions of diversified samples coming from various source combinations. Results are shown in Figure 4, which shows that more than 80% diversified samples have their responses obtained via model predicting, as token editing sets a strict condition that overlapped tokens must exist. Phrase-level editing also contributes to more high-quality personas with satisfactory fluency and semantic novelty. Analysis of data curriculum We first compare other data curriculum variants to show the usefulness of training with the designed data curriculum. The following variants are included: 1) Original : only the original dataset D (the hard curriculum in D 3 ) is used, which is equal to the base model; 2) Only augment : only the augmented dataset D a (the easy curriculum in D 3 ) is used; 3) Shuffle : shuffling of the original dataset D and the augmented dataset D a together to train the model; 4) Reverse : using the curricula in a reverse order, which means the hard curriculum first and then the easy one. Relevant results are shown in Table 4. There is no doubt that our curriculum is the best when comprehensively considering all aspects. Although Only augment and Reverse show high C-scores, their responses are much worse in n-gram accuracy as they involve more persona information while focusing less on the dialogue coherence during generating. Shuffle shows better performance than Original as it includes more augmented data than the original dataset, which may benefit the training. However, such a mixing strategy is not so efficient as our data curriculum as it neglects the learning difficulty of different data sources. Next, we further quantify the effect of curriculum training on models using the attention from the response on the persona sentences. We define two metrics, token-level/sentence-level consistent attention weight ( a t and a s ), to measure how the attention contributes to reflecting the proper personas. Recall that we concatenate the persona sentences and history utterances as a single model input. We record the token positions of the entailed persona sentences in the input sequence, which are determined by our NLI model, denoted as S . Then for each index s  S , if its corresponding token in the input also occurs in the response, we put this 7991 0 1 2 3 4 5 a t /  10 2 0 1 2 3 4 5 0 5 10 15 20 25 30 layer-2 0 5 10 15 20 25 a s /  10 2 layer-4 0 5 10 15 20 25 layer-6 0 5 10 15 20 25 Orig. index pair into a set T = { ( s, l ) } , where s and l are the token positions in the input sequence and response sequence respectively. Then we have two measurements for each sample: a t = 1 |T | (cid:88) ( i,j ) T a ij , a s = 1 YY (cid:88) i =1 (cid:88) j S a ij , (3) where a ij  [0 , 1] is the normalized scalar attention weight at the i -th decoding step on the j -th input token, i.e. (cid:80) j a ij = 1 , and Y is the length of the generated response. A higher a t / a s indicates that the model poses more attention on proper persona tokens, where the former one is fine-grained for reflecting how the attention works properly at each step, while the latter one is coarse-grained for the whole generated response. Part of the results with selected TRANSFORMER layers for these two metrics on all samples from the PersonaChat dev set are shown in Figure 5 (Refer to Appendix C.4 for the complete results). Obviously, our method shows the highest a t and a s on all given layers compared to other two curriculum variants. Such a superiority is more significant in higher layers, which is more decisive for generating responses (Fan et al., 2019). While the attentions weights tend to distribute uniformly in lower layers, which are close to the uniform values. Case study Some response samples generated when using TRANSFORMER as the base model are shown in Figure 6. Here H indicates dialogue history, a persona sentence shaded in a darker color denotes that it has a higher attention weight posed by the model. Our method D 3 can offer a model with the capability to pose more attention on the i love running , it is a stress reliever. proper persona texts during generating responses. More cases can be found in Appendix C.6. Our work targets the challenging personal-based dialogue generation task. Unlike previous work that designs a new dialogue model to improve the generation performance, we analyze the data issues affecting current models. On one hand, the data scale and diversity are expensive to increase by data collection. On the other hand, current data are difficult to learn with. Based on such an understanding, we propose a model-agnostic data manipulation method for this task. It first distills the original data and then augments both the amount and diversity of the distilled data. A curriculum training is then applied to utilize both augmented and original data. Experimental results showed that our method effectively improves the performance of two strong dialogue models, i.e. Transformer encoder-decoder and GPT2. We would like to thank Piji Li and Lemao Liu for their helpful discussion and feedback. We also thank anonymous reviewers for their constructive comments.
abstain abstain objective objective abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method objective abstain abstain method objective abstain method other other other other other other other other other other other other other other other other other other objective method abstain abstain abstain abstain abstain abstain method method method method method method abstain method abstain abstain abstain abstain method method abstain abstain abstain method method method method method abstain abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method method abstain method method abstain abstain abstain method abstain abstain method method method abstain abstain method method method method abstain abstain method method abstain method abstain abstain abstain abstain method method method abstain abstain abstain abstain abstain result other abstain abstain abstain other abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain other abstain method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain objective abstain abstain result other other