Neural Chat Translation (NCT) aims to translate conversational text into different languages. Existing methods mainly focus on modeling the bilingual dialogue characteristics ( e.g. , coherence) to improve chat translation via multi-task learning on small-scale chat translation data. Although the NCT models have achieved impressive success, it is still far from satisfactory due to insufficient chat translation data and simple joint training manners. To address the above issues, we propose a scheduled multi-task learning framework for NCT. Specifically, we devise a three-stage training framework to incorporate the large-scale in-domain chat translation data into training by adding a second pre-training stage between the original pre-training and fine-tuning stages. Further, we investigate where and how to schedule the dialogue-related auxiliary tasks in multiple training stages to effectively enhance the main chat translation task. Extensive experiments on four language directions (English  Chinese and English  German) verify the effectiveness and superiority of the proposed approach. Additionally, we will make the large-scale in-domain paired bilingual dialogue dataset publicly available for the research community. 1 1 Introduction A cross-lingual conversation involves speakers in different languages ( e.g. , one speaking in Chinese and another in English), where a chat translator can be applied to help them communicate in their native languages. The chat translator bilaterally converts the language of bilingual conversational text, e.g. from Chinese to English and vice versa (Wang et al., 2016a; Farajian et al., 2020; Liang et al., 2021a, 2022). Generally, since the bilingual dialogue corpus is scarce, researchers (Bao et al., 2020; Wang et al., 2020; Liang et al., 2021a,d) resort to making use of the large-scale general-domain data through the pre-training-then-fine-tuning paradigm as done in many context-aware neural machine translation models (Tiedemann and Scherrer, 2017; Maruf and Haf-fari, 2018; Miculicich et al., 2018; Tu et al., 2018; Voita et al., 2018, 2019a,b; Yang et al., 2019; Wang et al., 2019; Maruf et al., 2019; Ma et al., 2020, etc), having made significant progress. However, conventional pre-training on large-scale general-domain data usually learns general language patterns, which is also aimless for capturing the useful dialogue context to chat translation, and fine-tuning usually suffers from insufficient supervised data (about 10k bilingual dialogues). Some studies (Gu et al., 2020; Gururangan et al., 2020; Liu et al., 2021; Moghe et al., 2020; Wang et al., 2020; Ruder, 2021) have shown that learning domain-specific patterns by additional pre-training is beneficial to the models. To this end, we firstly construct the large-scale in-domain chat translation data 2 . And to 2 Firstly, to build the data, for English  Chinese (En  Zh), we crawl two consecutive English and Chinese movie subtitles (not aligned). For English  German (En  De), we download two consecutive English and German movie subtitles (not aligned). Then, we use several advanced technologies to align En  Zh and En  De subtitles. Finally, we obtain the paired bilingual dialogue dataset. Please refer to  3.1 for details. incorporate it for learning domain-specific patterns, we then propose a three-stage training framework via adding a second pre-training stage between general pre-training and fine-tuning, as shown in Fig. 1. To further improve the chat translation performance through modeling dialogue characteristics ( e.g. , coherence), inspired by previous studies (Phang et al., 2020; Liang et al., 2021d; Pruk-sachatkun et al., 2020), we incorporate several dialogue-related auxiliary tasks to our three-stage training framework. Unfortunately, we find that simply introducing all auxiliary tasks in the conventional multi-task learning manner does not obtain significant cumulative benefits as we expect. It indicates that the simple joint training manner may limit the potential of these auxiliary tasks, which inspires us to investigate where and how to make these auxiliary tasks work better for the main NCT task. To address the above issues, we present a S cheduled M ulti-task L earning framework (SML) for NCT, as shown in Fig. 1. Firstly, we propose a three-stage training framework to introduce our constructed in-domain chat translation data for learning domain-specific patterns. Secondly, to make the most of auxiliary tasks for the main NCT task, where : we analyze in which stage these auxiliary tasks work well and find that they are different strokes for different folks . Therefore, to fully exert their advantages for enhancing the main NCT task, how : we design a gradient-based strategy to dynamically schedule them at each training step in the last two training stages, which can be seen as a fine-grained joint training manner. In this way, the NCT model is effectively enhanced to capture both domain-specific patterns and dialogue-related characteristics ( e.g. , coherence) in conversation, which thus can generate better translation results. We validate our SML framework on two datasets: BMELD (Liang et al., 2021a) (En  Zh) and BConTrasT (Farajian et al., 2020) (En  De). Experimental results show that our model gains consistent improvements on four translation tasks in terms of both BLEU (Papineni et al., 2002) and TER (Snover et al., 2006) scores, demonstrating its effectiveness and generalizability. Human evaluation further suggests that our model can produce more coherent and fluent translations compared to the previous related methods. Our contributions are summarized as follows:  We propose a scheduled multi-task learning framework with three training stages, where a gradient-based scheduling strategy is designed to fully exert the auxiliary tasks' advantages for the main NCT task, for higher translation quality. Extensive experiments on four chat translation tasks show that our model achieves new state-of-the-art performance and outperforms the existing NCT models by a significant margin. We contribute two large-scale in-domain paired bilingual dialogue corpora (28M for En  Zh and 18M for En  De) to the research community. We introduce the conventional multi-task learning framework (Liang et al., 2021d) for NCT, which includes four parts: problem formalization ( 2.1), the NCT model ( 2.2), existing three auxiliary tasks ( 2.3), and training objective ( 2.4). In a bilingual conversation, we assume the two speakers have alternately given utterances in different languages for u turns, resulting in X 1 , X 2 , X 3 , ..., X u and Y 1 , Y 2 , Y 3 , ..., Y u on the source and target sides, respectively. Among these utterances, X 1 , X 3 , X 5 , ..., X u are originally spoken and Y 1 , Y 3 , Y 5 , ..., Y u are the corresponding translations in the target language. Similarly, Y 2 , Y 4 , Y 6 , ..., Y u  1 are originally spoken and X 2 , X 4 , X 6 , ..., X u  1 are the translated utterances in the source language. According to languages, we define the dialogue history context of X u on the source side as CX u ={ X 1 , X 2 , X 3 , ..., X u  1 } and that of Y u on the target side as CY u ={ Y 1 , Y 2 , Y 3 , ..., Y u  1 } . 3 The goal of an NCT model is to translate X u to Y u with dialogue history context CX u and CY u . The NCT model (Ma et al., 2020; Liang et al., 2021d) utilizes the standard transformer (Vaswani et al., 2017) architecture with an encoder and a decoder 4 . 3 For each of { CX u , CY u }, we add the special token [CLS]' tag at the head of it and use another token [SEP]' to delimit its included utterances, as in Devlin et al. (2019). 4 Here, we just describe some adaptions to the NCT model, and please refer to Vaswani et al. (2017) for more details. In the encoder, it takes [ CX u ; X u ] as input, where [; ] denotes the concatenation. The input embedding consists of word embedding WE , position embedding PE , and turn embedding TE : B ( x i ) = WE ( x i ) + PE ( x i ) + TE ( x i ) , where WE  R | V | d and TE  R | T | d . 5 When computation in the encoder, words in CX u can only be attended by those in X u at the first encoder layer while CX u is masked at the other layers, which is the same implementation as in Ma et al. (2020). In the decoder, at each decoding time step t , the top-layer ( L -th) decoder hidden state h Ld,t is fed into a softmax layer to predict the probability distribution of the next target token: p ( Y u,t | Y u,<t , X u , CX u ) = Softmax( W o h Ld,t + b o ) , where Y u,<t denotes the preceding tokens before the t -th time step in the utterance Y u , W o  R | V | d and b o  R | V | are trainable parameters. To generate coherent translation, Liang et al. (2021d) present Monolingual Response Generation (MRG) task, Cross-lingual Response Generation (XRG) task, and Next Utterance Discrimination (NUD) task during the NCT model training. MRG. Given the dialogue context CY u in the target language, it forces the NCT model to generate the corresponding utterance Y u coherent to CY u . Particularly, the encoder of the NCT model is used to encode CY u , and the NCT decoder predicts Y u . The training objective of this task is formulated as: LMRG =  | Y u | (cid:88) t =1 log( p ( Y u,t |C Y u , Y u,<t )) , p ( Y u,t |C Y u , Y u,<t ) = Softmax( W m h Ld,t + b m ) , where h Ld,t is the L -th decoder hidden state at the t -th decoding step, W m and b m are trainable parameters. XRG. Similar to MRG, the NCT model is also jointly trained to generate the corresponding utterance Y u which is coherent to the given dialogue 5 | V | , | T | and d denote the size of shared vocabulary, maximum dialogue turns, and the hidden size, respectively. history context CX u in the source language: LXRG =  | Y u | (cid:88) t =1 log( p ( Y u,t |C X u , Y u,<t )) , p ( Y u,t |C X u , Y u,<t ) = Softmax( W c h Ld,t + b c ) , where W c and b c are trainable parameters. NUD. The NUD task aims to distinguish whether the translated text is coherent to be the next utterance of the given dialogue history context. Specifically, the positive and negative samples are firstly constructed: (1) the positive sample ( CY u , Y u + ) with the label  = 1 consists of the target utterance Y u and its dialogue history context CY u ; (2) the negative sample ( CY u , Y u  ) with the label  = 0 consists of the identical CY u and a randomly selected utterance Y u  from the preceding context of Y u . Formally, the training objective of NUD is defined as follows: LNUD =  log( p (  = 1 |C Y u , Y u + ))  log( p (  = 0 |C Y u , Y u  )) , p (  =1 |C Y u , Y u )=Softmax( W n [ HY u ; HC Yu ]) , where HY u and HC Yu denote the representations of the target utterance Y u and CY u , respectively. Concretely, HY u is calculated as 1 | Y u | (cid:80) | Y u | t =1 h Le,t while HC Yu is defined as the encoder hidden state h Le, 0 of the prepended special token [CLS]' of CY u . W n is the trainable parameter of the NUD classifier and the bias term is omitted for simplicity. With the main chat translation task and three auxiliary tasks, the total training objective of the conventional multi-task learning is formulated as: L = LNCT +  ( LMRG + LXRG + LNUD ) , (2) In this section, we introduce the proposed S cheduled M ulti-task L earning (SML) framework, including three stages: general pre-training, in-domain pre-training, and in-domain fine-tuning, as shown in Fig. 1. Specifically, we firstly describe the process of in-domain pre-training ( 3.1) and then present some findings of conventional multi-task learning ( 3.2), which inspire us to investigate the scheduled multi-task learning ( 3.3). Finally, we 4377 elaborate on the process of training and inference ( 3.4). For the second in-domain pre-training, we firstly build an in-domain paired bilingual dialogue data and then conduct pre-training on it. To construct the paired bilingual dialogue data, we firstly crawl the in-domain consecutive movie subtitles of En  Zh and download the consecutive movie subtitles of En  De on related websites 6 . Since both bilingual movie subtitles are not strictly aligned, we utilize the Vecalign tool (Thompson and Koehn, 2019), an accurate sentence alignment algorithm, to align them. Meanwhile, we leverage the LASER toolkit 7 to obtain the multilingual embedding for better alignment performance. Consequently, we obtain two relatively clean paired movie subtitles. According to the setting of dialogue context length in Liang et al. (2021a), we take four consecutive utterances as one dialogue, and then filter out duplicate dialogues. Finally, we attain two in-domain paired bilingual dialogue dataset, the statistics of which are shown in Tab. 1. Datasets # Dialogues # Utterances # Sentences En  Zh 28,214,769 28,238,877 22,244,006 En  De 18,041,125 18,048,573 45,541,367 Table 1: Statistics of our constructed chat translation data. Based on the constructed in-domain bilingual corpus, we continue to pre-train the NCT model after the general pre-training stage, and then go to the in-domain fine-tuning stage, as shown in the In-domain Pre-training&Fine-tuning parts of Fig. 1. 3.2 Findings of Conventional Multi-task Learning According to the finding that multi-task learning can enhance the NCT model (Liang et al., 2021d), in the last two training processes ( i.e. , the In-domain Pre-training and In-domain Fine-tuning parts of Fig. 1), we conduct extensive multi-task learning experiments, aiming to achieve a better NCT model. Firstly, we present one additional auxiliary task, i.e. Cross-lingual NUD (XNUD), given the intuition that more dialogue-related tasks may 6 En  Zh: https://www.kexiaoguo.com/ and En  De: https://opus.nlpl.eu/OpenSubtitles.php 7 https://github.com/facebookresearch/LASER 05* ;5* 18' ;18' $OO (Q=K5HVXOWVLQ'LIIHUHQW6WDJHV         % /( 8 6HFRQG6WDJH)LQHWXQLQJ6WDJH%RWK6WDJHV1&7PRGHOZRWDVN 05* ;5* 18' ;18' $OO =K(Q5HVXOWVLQ'LIIHUHQW6WDJHV       % /( 8 6HFRQG6WDJH)LQHWXQLQJ6WDJH%RWK6WDJHV1&7PRGHOZRWDVN Figure 2: The effect of each task on validation sets in different training stages, under transformer Base setting, where All denotes all four auxiliary tasks. yield better performance. Then, we conclude some multi-task learning findings that could motivate us to investigate how to use these auxiliary tasks well. XNUD. Similar to the NUD task described in  2.3, the XNUD aims to distinguish whether the translated text is coherent to be the next utterance of the given cross-lingual dialogue history context. Compared to the NUD task, the different point lies in the cross-lingual dialogue context history, i.e. , a positive sample ( CX u , Y u + ) with the label  = 1 and a negative sample ( CX u , Y u  ) with the label  = 0 . Formally, the training objective of XNUD is defined as follows: LXNUD =  log( p (  = 1 |C X u , Y u + ))  log( p (  = 0 |C X u , Y u  )) , p (  =1 |C X u , Y u )=Softmax( W x [ HY u ; HC Xu ]) , where HC Xu denotes the representation of CY u , which is calculated as same as HC Yu in NUD. W x is the trainable parameter of the XNUD classifier and the bias term is omitted for simplicity. Findings. Based on four auxiliary tasks (MRG, XRG, NUD, and XNUD), we investigate in which stage in Fig. 1 the auxiliary tasks work well in a conventional multi-task learning manner 8 and the following is what we find from Fig. 2:  Each auxiliary task can always bring improvement compared with the NCT model w/o task; 8 Note that, in the last two in-domain stages, we use the conventional multi-task learning to pre-train and fine-tune models rather than the scheduled multi-task learning. By contrast, XRG and XNUD tasks perform relatively poorly in the final fine-tuning stage than MRG and NUD tasks;  Some tasks used only in one stage ( e.g. , XRG and XNUD in the second stage) perform better than being used in both stages, revealing that different auxiliary tasks may prefer different stages to exert their advantages; (one best setting seems that all tasks are used in the second stage while only MRG and NUD tasks are used in the final fine-tuning stage.)  Using all auxiliary tasks in a conventional multi-task learning manner does not obtain significant cumulative benefits. Inspired by Yu et al. (2020), we design a gradient-based scheduled multi-task learning algorithm to dynamically schedule all auxiliary tasks at each training step, as shown in Algorithm 1. Specifically, at each training step ( line 1 ), for each task we firstly compute its gradient to model parameters  ( lines 2  4 , and we denote the gradient of the main NCT task as g nct ). Then, we obtain the projection of the gradient g k of each auxiliary task k onto g nct ( line 5 ), as shown in Fig. 3. Finally, we utilize the sum of g nct and all projection ( i.e. , the blue arrows part, as shown in Fig. 3) of auxiliary tasks to update model parameters. The core ideas behind the gradient-based SML algorithm are: (1) when the cosine similarity between g k and g nct is positive, i.e. , the gradient projection g  k is in the same gradient descent direction with the main NCT task, i.e. , Fig. 3 (a), which could help the NCT model achieve optimal solution; (2) when the cosine similarity between g k and g nct is negative, i.e. , Fig. 3 (b), which can avoid the model being optimized too fast and overfitted. Therefore, we also keep the inverse gradient to prevent the NCT model from overfitting as a regularizer. In this way, such auxiliary task joins in training at each step with the NCT task when its gradient projection is in line with g nct , which acted as a fine-grained joint training manner. Our training process includes three stages: the first pre-training stage on the general-domain sentence the second in-domain pre-training stage, and the final in-domain fine-tuning stage on the chat translation data: J = LNCT +  T (cid:88) k L k , (4) where T is the auxiliary tasks set and we keep the balancing hyper-parameter  . Although the form of L k is the same with Eq. 2, the gradient that participates in updating model parameters is different where it depends on the gradient descent direction of the NCT task in Eq. 4. At inference, all auxiliary tasks are not involved and only the NCT model after scheduled multi-task fine-tuning is applied to chat translation. Datasets. The training of our SML framework consists of three stages: (1) pre-train the model on a large-scale sentence-level NMT corpus (WMT20 9 ); 9 http://www.statmt.org/wmt20/translation-task.html 4379 Models En  Zh Zh  En En  De De  En BLEU  TER  BLEU  TER  BLEU  TER  BLEU  TER  Base Trans. (2) further pre-train the model on our constructed in-domain chat translation corpus; (3) fine-tune on the target chat translation corpus: BMELD (Liang et al., 2021a) and BConTrasT (Farajian et al., 2020). The target dataset details ( e.g. , splits of training, validation or test sets) are described in Appendix A. Metrics. Following Liang et al. (2021d), we use SacreBLEU 10 (Post, 2018) and TER (Snover et al., 2006) with the statistical significance test (Koehn, 2004) for fair comparison. Specifically, we report character-level BLEU for En  Zh, case-insensitive BLEU score for Zh  En, and case-sensitive BLEU score likewise for En  De. In this paper, we adopt the settings of standard Transformer-Base and Transformer-Big in Vaswani et al. (2017). Generally, we utilize the settings in Liang et al. (2021d) for fair comparison. For more details, please refer to Appendix B. We investigate the effect of the XNUD task in  5.4, where the new XNUD performs well based on existing auxiliary tasks. 10 BLEU+case.mixed+numrefs.1+smooth.exp+tok.13a+ version.1.4.13 4.3 Comparison Models Sentence-level NMT Systems. Trans. w/o FT and Trans. (Vaswani et al., 2017): both are the de-facto transformer-based NMT models, and the difference is that the Trans. model is fine-tuned on the chat translation data after being pre-trained on sentence-level NMT corpus. Context-aware NMT Systems. Dia-Trans. (Maruf et al., 2018): A Transformer-based model where an additional encoder is used to introduce the mixed-language dialogue history, re-implement by Liang et al. (2021a). Gate-Trans. (Zhang et al., 2018) and NCT (Ma et al., 2020): Both are document-level NMT Transformer models where they introduce the dialogue history by a gate and by sharing the first encoder layer, respectively. CPCC (Liang et al., 2021a): A variational model that focuses on incorporating dialogue characteristics into a translator for better performance. In Tab. 2, We report the main results on En  Zh and En  De under Base and Big settings. In Tab. 3, we present additional results on En  Zh. Results on En  Zh. Under the Base setting, our model significantly outperforms the sentence-level/context-aware baselines by a large margin ( e.g. , the previous best CSA-NCT), 4.58  on En  Zh and 4.06  on Zh  En, showing the effectiveness of the large-scale in-domain data and our scheduled multi-task learning. In terms of TER, SML also performs best on the two directions, 5.0  and 4.3  than CPCC (the lower the better), respectively. Under the Big setting, our model consistently surpasses all existing systems once again. Results on En  De. On both En  De and De  En, our model presents notable improvements over all comparison models by up to 2.50  and 2.69  BLEU gains under the Base setting, and by 2.55  and 2.53  BLEU gains under the Big setting, respectively. These results demonstrate the superiority of our three-stage training framework and also show the generalizability of our model across different language pairs. Since the baselines of En  De are very strong, the results of En  De are not so significant than En  Zh. Additional Results. Tab. 2 presents our overall model performance, though, strictly speaking, it is unfair to directly compare our approaches with previous ones. Therefore, we conduct additional experiments in Tab. 3 under two settings: ( i ) using the original pre-training-then-fine-tuning framework without introducing the large-scale in-domain data ( i.e. , Two-stage w/o data group); ( ii ) using the proposed three-stage method with the large-scale in-domain data ( i.e. , Three-stage w/ data group). And we conclude that (1) the same model ( e.g. , SML) can be significantly enhanced by the second in-domain pre-training stage, demonstrating the effectiveness of the second pre-training on the in-domain data; (2) our SML model always exceeds the conventional multi-task learning model M-NCT in both settings, indicating the superiority of the scheduled multi-task learning strategy. We conduct ablation studies in Tab. 4 and Tab. 5 to answer the following two questions. Q1 : why a three-stage training framework? and Q2 : why the scheduled multi-task learning strategy? To answer Q1 , in Tab. 4, we firstly investigate the effect of the large-scale in-domain chat translation data and further explore where to use it. Firstly, the results of rows 1  3 substantially outperform those in row 0, proving the availability of incorporating the in-domain data. Secondly, the results of 4381 # Training Manners? row 3 significantly surpass rows 1  2, indicating that the in-domain data used in the proposed second stage of our three-stage training framework is very successful rather than used in the stage of pre-training-then-fine-tuning paradigm. That is, the experiments show the effectiveness and necessity of our three-stage training framework. To answer Q2 , we investigate multiple multitask learning strategies in Tab. 5. Firstly, the results of row 3 are notably higher than those of rows 0  2 in both language directions, obtaining significant cumulative benefits of auxiliary tasks than rows 0  2, demonstrating the validity of the proposed SML strategy. Secondly, the results of row 3 vs row 4 show that the inverse gradient projection of auxiliary tasks also has a positive impact on the model performance, which may prevent the model from overfitting, working as a regularizer. All experiments show the superiority of our scheduled multi-task learning strategy. Inspired by Bao et al. (2020) and Liang et al. (2021a), we use two criteria for human evaluation to judge whether the translation is: 1. semantically coherent with the dialogue history? 2. fluent and grammatically correct? Firstly, we randomly sample 200 conversations from the test set of BMELD in En  Zh. Then, we use 6 models in Tab. 6 to generate translated utterances of these sampled conversations. Finally, we assign the translated utterances and their corre-Models ( Base ) Coherence Fluency Trans. sponding dialogue history utterances in the target language to three postgraduate human annotators, and then ask them to make evaluations (0/1 score) according to the above two criteria, and average the scores as the final result. Tab. 6 shows that our model generates more coherent and fluent translations when compared with other models (significance test, p < 0.05), which shows the superiority of our model. The inter-annotator agreements calculated by the Fleiss' kappa (Fleiss and Cohen, 1973) are 0.558 and 0.583 for coherence and fluency, respectively. It indicates Moderate Agreement for both criteria. s i is the sentence. We use Word2Vec 11 (Mikolov et al., 2013) trained on a dialogue dataset 12 to obtain the distributed word vectors whose dimension is set to 100. Tab. 7 shows the measured coherence of different models on validation set of BMELD in En  Zh direction. It shows that our SML produces more coherent translations compared to all existing models (significance test, p < 0.01). We investigate the effect of the XNUD task. As shown in Tab. 8, the M-NCT denotes the multitask learning model jointly trained with four auxiliary tasks in conventional manner. After removing the XNUD task, the performance drops to some extend, indicating that the new XNUD task achieves further performance improvement based on three existing auxiliary tasks (Liang et al., 2021d). Then, based on the strong M-NCT model, we further investigate where and how to make the most of them for the main NCT task. Neural Chat Translation. The goal of NCT is to train a dialogue-aware translation model using the bilingual dialogue history, which is different from document-level/sentence-level machine translation (Maruf et al., 2019; Ma et al., 2020; Yan et al., 2020; Meng and Zhang, 2019; Zhang et al., 2019). Previous work can be roughly divided into two categories. One (Wang et al., 2016b; Maruf et al., 2018; Zhang and Zhou, 2019; Rikters et al., 2020) mainly pays attention to automatically constructing the bilingual corpus since no publicly available human-annotated data (Farajian et al., 2020). The other (Wang et al., 2021; Liang et al., 2021a,d) aims to incorporate the bilingual dialogue characteristics 11 https://code.google.com/archive/p/word2vec/ 12 We choose our constructed dialogue corpus to learn the word embedding. into the NCT model via multi-task learning. Different from the above studies, we focus on introducing the in-domain chat translation data to learn domain-specific patterns and scheduling the auxiliary tasks to exert their potential for high translation quality. Multi-task Learning. Conventional multi-task learning (MTL) (Caruana, 1997), which trains the model on multiple related tasks to promote the representation learning and generalization performance, has been successfully used in many NLP tasks (Collobert and Weston, 2008; Ruder, 2017; Deng et al., 2013; Liang et al., 2021c,b). In the NCT, conventional MTL has been explored to inject the dialogue characteristics into models with dialogue-related tasks such as response generation (Liang et al., 2021a,d). In this work, we instead focus on how to schedule the auxiliary tasks at training to make the most of them for better translations. This paper proposes a scheduled multi-task learning framework armed with an additional in-domain pre-training stage and a gradient-based scheduled multi-task learning strategy. Experiments on En  Zh and En  De demonstrate that our framework significantly improves translation quality on both BLEU and TER metrics, showing its effectiveness and generalizability. Human evaluation further verifies that our model yields better translations in terms of coherence and fluency. Furthermore, we contribute two large-scale in-domain paired bilingual dialogue datasets to the research community. The research work descried in this paper has been supported by the National Key R&D Program of China (2020AAA0108001) and the National Nature Science Foundation of China (No. 61976015, 61976016, 61876198 and 61370130). Liang is supported by 2021 Tencent Rhino-Bird Research Elite Training Program. The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve this paper.
abstain abstain abstain objective method objective abstain abstain abstain abstain other abstain abstain method method abstain method result abstain objective result result objective objective result result abstain abstain abstain method objective objective objective abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain other abstain abstain abstain abstain other abstain abstain abstain abstain abstain other other abstain abstain abstain abstain other abstain method method method abstain abstain method abstain method other other abstain abstain abstain abstain abstain other abstain abstain other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain other method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other method other other other method objective objective abstain objective other other other