Back-translation is a critical component of Unsupervised Neural Machine Translation (UNMT), which generates pseudo parallel data from target monolingual data. A UNMT model is trained on the pseudo parallel data with translated source , and translates natural source sentences in inference. The source discrepancy between training and inference hinders the translation performance of UNMT models. By carefully designing experiments, we identify two representative characteristics of the data gap in source: (1) style gap (i.e., translated vs. natural text style) that leads to poor generalization capability; (2) content gap that induces the model to produce hallucination content biased towards the target language. To narrow the data gap, we propose an online self-training approach, which simultaneously uses the pseudo parallel data { natural source, translated target } to mimic the inference scenario. Experimental results on several widely-used language pairs show that our approach outperforms two strong baselines (XLM and MASS) by remedying the style and content gaps. 1 1 Introduction In recent years, there has been a growing interest in unsupervised neural machine translation (UNMT), which requires only monolingual corpora to accomplish the translation task (Lample et al., 2018a,b; Artetxe et al., 2018b; Yang et al., 2018; Ren et al., 2019). The key idea of UNMT is to use back-translation (BT) (Sennrich et al., 2016) to construct  Work was done when Zhiwei He was interning at Tencent AI Lab. the pseudo parallel data for translation modeling. Typically, UNMT back-translates the natural target sentence into the synthetic source sentence (trans-lated source) to form the training data. A BT loss is calculated on the pseudo parallel data { translated source, natural target } to update the parameters of UNMT models. In Supervised Neural Machine Translation (SNMT), Edunov et al. (2020) found that BT suffers from the translationese problem (Zhang and Toral, 2019; Graham et al., 2020) in which BT improves BLEU score on the target-original test set with limited gains on the source-original test set. Unlike authentic parallel data available in the SNMT training data, the UNMT training data entirely comes from pseudo parallel data generated by the back-translation. Therefore in this work, we first revisit the problem in the UNMT setting and start our research from an observation (2): with comparable translation performance on the full test set, the BT based UNMT models achieve better translation performance than the SNMT model on the target-original (i.e. translationese) test set, while achieves worse performance on the source-6611 original ones. In addition, the pseudo parallel data { translated source, natural target } generated by BT poses great challenges for UNMT, as shown in Table 1. First, there exists the input discrepancy between the translated source (translated style) in UNMT training data and the natural source (natural style) in inference data. We find that the poor generalization capability caused by the style gap (i.e., translated style v.s natural style) limited the UNMT translation performance (3.1). Second, the translated pseudo parallel data suffers from the language coverage bias problem (Wang et al., 2021), in which the content of UNMT training data biases towards the target language while the content of the inference data biases towards the source language. The content gap results in hallucinated translations (Lee et al., 2018; Wang and Sennrich, 2020) biased towards the target language (3.2). To alleviate the data gap between the training and inference, we propose an online self-training (ST) approach to improve the UNMT performance. Specifically, besides the BT loss, the proposed approach also synchronously calculates the ST loss on the pseudo parallel data { natural source, translated target } generated by self-training to update the parameters of UNMT models. The pseudo parallel data { natural source, translated target } is used to mimic the inference scenario with { natural source, translated target } to bridge the data gap for UNMT. It is worth noting that the proposed approach does not cost extra computation to generate the pseudo parallel data { natural source, translated target } 2 , which makes the proposed method efficient and easy to implement. We conduct experiments on the XLM (Lample and Conneau, 2019) and MASS (Song et al., 2019) UNMT models on multiple language pairs with varying corpus sizes (WMT14 En-Fr / WMT16 EnDe / WMT16 En-Ro / WMT20 En-De / WMT21 En-De). Experimental results show that the proposed approach achieves consistent improvement over the baseline models. Moreover, we conduct extensive analyses to understand the proposed approach better, and the quantitative evidence reveals that the proposed approach narrows the style and content gaps to achieve the improvements. 2 The vanilla UNMT model adopts the dual structure to train both translation directions together, and the pseudo parallel data { natural source, translated target } has already been generated and is used to update the parameters of UNMT model in the reverse direction. We empirically analyze the data gap between training and inference for UNMT and identify two critical factors: style gap and content gap. Our empirical study demonstrates that the back-translation based UNMT framework suffers from the translationese problem, causing the inaccurate evaluation of UNMT models on standard benchmarks. We propose a simple and effective approach for incorporating the self-training method into the UNMT framework to remedy the data gap between the training and inference. Notations. Let X and Y denote the language pair, and let X = { x i } Mi =1 and Y = { y j } Nj =1 represent the collection of monolingual sentences of the corresponding language, where M, N are the size of the corresponding set. Generally, UNMT method that based on BT adopts dual structure to train a bidirectional translation model (Artetxe et al., 2018b, 2019; Lample et al., 2018a,b). For the sake of simplicity, we only consider translation direction X  Y unless otherwise stated. Online BT. Current mainstream of UNMT methods turn the unsupervised task into the synthetic supervised task through BT, which is the most critical component in UNMT training. Given the translation task X  Y where target corpus Y is available, for each batch, the target sentence y  Y is used to generate its synthetic source sentence by the backward model MTY  X : x  = arg max x PY  X ( x | y ;   ) , (1) where   is a fixed copy of the current parameters  indicating that the gradient is not propagated through   . In this way, the synthetic parallel sentence pair { x  , y } is obtained and used to train the forward model MTX  Y in a supervised manner by minimizing: LB = E y Y [  log PX  Y ( y | x  ;  )] . It is worth noting that the synthetic sentence pair generated by the BT is the only supervision signal of UNMT training. Objective function. In addition to BT, denoising auto-encoding (DAE) is an additional loss term of UNMT training, which is denoted by LD and is not the main topic discussed in this work. In all, the final objective function of UNMT is: L = LB +  DLD , (3) where  D is the hyper-parameter weighting DAE loss term. Generally,  D starts from one and decreases as the training procedure continues 3 . To verify whether the UNMT model suffers from the input gap between training and inference and thus is biased towards translated input while against natural input, we conduct comparative experiments between SNMT and UNMT models. Setup We evaluate the UNMT and SNMT models on WMT14 En-Fr, WMT16 En-De and WMT16 En-Ro test sets, following Lample and Conneau (2019) and Song et al. (2019). We first train the UNMT models on the above language pairs with model parameters initialized by XLM and MASS models. Then, we train the corresponding SNMT models whose performance on the full test sets is controlled to be approximated to UNMT by undersampling training data. Finally, we evaluate the UNMT and SNMT models on the target-original and source-original test sets, whose inputs are translated and natural respectively. Unless otherwise stated, we follow previous work (Lample and Conneau, 2019; Song et al., 2019) to use case-sensitive BLEU score (Papineni et al., 2002) with the multi-bleu.perl 4 script as the evaluation metric. Please refer to Appendix B for the results of SacreBLEU, and refer to Appendix A for the training details of SNMT and UNMT models. Results We present the translation performance in terms of the BLEU score in Table 2 and our observations are:  UNMT models perform close to the SNMT models on the full test sets with 0.3 BLEU difference at most on average (33.5/33.9 vs. 33.6). UNMT models outperform SNMT models on target-original test sets (translated input) with 3 Verified from open-source XLM Github implementation. 4.4 BLEU points (36.8/37.6 vs. 33.2). UNMT models underperform the SNMT models on source-original test sets (natural input) with an average performance degradation of 4.4 and 4.2 BLUE points (28.7/28.9 vs. 33.1). The above observations are invariant concerning the pre-trained model and translation direction. In particular, the unsatisfactory performance of UNMT under natural input indicates that UNMT is overestimated on the previous benchmark. We attribute the phenomenon to the data gap between training and inference for UNMT: there is a mismatch between natural inputs of source-original test data and the back-translated inputs that UNMT employed for training. This work focuses on the experiments on the source-original test sets (i.e., the input of an NMT translation system is generally natural), which is closer to the practical scenario. 5 3 Data Gap between Training and Inference In this section, we identity two representative data gaps between training and inference data for 5 From WMT19, the WMT community proposes to use the source-original test with natural input sets to evaluate the translation performance. UNMT: style gap and content data. We divide the test sets into two portions: the natural input portion with source sentences originally written in the source language and the translated input portion with source sentences translated from the target language. Due to the limited space, we conduct the experiments with pre-trained XLM initialization and perform analysis with different kinds of inputs (i.e., natural and translated inputs) on De  En newstest2013-2018 unless otherwise stated. To perform the quantitative analysis of the style gap, we adopt KenLM 6 to train a 4-gram language model on the UNMT translated source sentences 7 and use the language model to calculate the perplexity (PPL) of natural and translated input sentences in the test sets. The experimental results are shown in Table 3. The lower perplexity value (219 < 242) indicates that compared with the natural inputs, the UNMT translated training inputs have a more similar style with translated inputs in the test sets . In order to further reveal the influence of the style gap on UNMT, we manually eliminated it and re-evaluated the models on the natural input portion of WMT16 De  En. Concretely, We first take the third-party Google Translator to translate 6 https://github.com/kpu/kenlm 7 To alleviate the content bias problem, we generate the training data 50% from En  De translation and 50% from round trip translation De  En  De. the target English sentences of the test sets into the source German language to eliminate the style gap. And then we conduct translation experiments on the natural input portion and its Google translated portion to evaluate the impact of the style gap on the translation performance. We list the experimental results in Table 4. We can find that by converting from the natural inputs (natural De) to the translated inputs (translated De  ), the UNMT model achieves more improvement than the SNMT model (-2.8 > -6.3), demonstrating that the style gap inhibits the UNMT translation output quality. In this section, we show the existence of the content gap by (1) showing the most high-frequency name entities, (2) calculating content similarity using term frequency-inverse document frequency (TF-IDF) for the training and inference data. We use spaCy 8 to recognize German named entities for the UNMT translated source sentences, natural inputs and translated inputs in test sets, and show the ten most frequent name entities in Table 5. From the table, we can observe that the UNMT translated source sentences have few named entities biased towards source language German (words in red color), while having more named entities biased towards target language English, e.g., USA, Obama. It indicates that the content of the UNMT translated source sentences is biased towards the target language English. Meanwhile, the natural input portion of the inference data has more named entities biased towards source language German (words in red color), demonstrating that the content gap exists between the natural input portion of the inference data and the UNMT translated training data. Next, we remove the stop words and use the term frequency-inverse document frequency (TF-IDF) approach to calculate the content similarity between the training and inference data. Similarity scores are presented in Table 6. We can observe that the UNMT translated source data has a more significant similarity score with translated inputs which are generated from the target English sentences. This result indicates that the content of UNMT translated source data is more biased towards the target language , which is consistent with the findings in Table 5. As it is difficult to measure the name entities 8 https://github.com/explosion/spaCy 6614 Data Most Frequent Name Entities Natural Infer. translation accuracy in terms of BLEU evaluation metric, we provide a translation example in Table 7 to show the effect of the content gap in the UNMT translations (more examples in Appendix C). We observe that the UNMT model outputs the hallucinated translation U.S., which is biased towards the target language English. We present a quantitative analysis to show the impact of the content gap on UNMT translation performance in Section 6.2. To bridge the data gap between training and inference of UNMT, we propose a simple and effective method through self-training. For the translation task X  Y , we generate the source-original training samples from the source corpus X to improve the model's translation performance on natural inputs. For each batch, we apply the forward model MTX  Y on the natural source sentence x to generate its translation: y  = arg max y PX  Y ( y | x ;   ) . where  S is the hyper-parameter weighting the self-training loss term. It is worth noting that the generation step of Eq.(4) has been done by the BT step of Y  X training. Thus, the proposed method will not increase the training cost significantly but make the most of the data generated by BT (Table 9). Data We follow the common practices to conduct experiments on several UNMT benchmarks: WMT14 En-Fr, WMT16 En-De, WMT16 En-Ro. The details of monolingual training data are delineated in Appendix A.2. We adopt En-Fr newsdev2014, En-De newsdev2016, En-Ro news-dev2016 as the validation (development) sets, and En-Fr newstest2014, En-De newstest2016, En-Ro newstest2016 as the test sets. In addition to the full test set, we split the test set into two parts: target-original and source-original, and evaluate the model's performance on the three kinds of test sets. We use the released XLM BPE codes and vocabulary for all language pairs. Model We evaluate the UNMT model fine-tuned on XLM 9 and MASS 10 pre-trained model (Lample and Conneau, 2019; Song et al., 2019). For XLM models, we adopt the pre-trained models released by Lample and Conneau (2019) for all language pairs. For MASS models, we adopt the pre-trained 9 https://github.com/facebookresearch/XLM 10 https://github.com/microsoft/MASS 6615 Testset Model Approach En-Fr En-De En-Ro Avg. models released by Song et al. (2019) for En-Fr and En-Ro and continue pre-training the MASS model of En-De for better reproducing the results. More details are delineated in Appendix A.2. Table 8 shows the translation performance of XLM and MASS baselines and our proposed models. We have the following observations:  Our re-implemented baseline models achieve comparable or even better performance as reported in previous works. The reproduced XLM+UNMT model has an average improvement of 1.4 BLEU points compared to the original report in Lample and Conneau (2019) and MASS+UNMT model is only 0.1 BLEU lower on average than Song et al. (2019). Our approach with online self-training significantly improves overall translation performance (+0.8 BLEU on average). This demonstrates the universality of the proposed approach on both large-scale (En-Fr, En-De) and data imbalanced corpus (En-Ro). In the translated input scenario, our approach achieves comparable performance to baselines. It demonstrates that although the sample of self-training is source-original style, our approach does not sacrifice the performance on the target-original side. In the natural input scenario, we find that our proposed approach achieves more significant improvements, with +1.1 and +1.3 average BLEU on both baselines. The reason is that the source-original style sample introduced by self-training alleviates model bias between natural and translated input. We compare online self-training with the following two related methods, which also incorporate natural inputs in training: Offline Self-training model distilled from the forward and backward translated data generated by the trained UNMT model. CBD (Nguyen et al., 2021) model distilled from the data generated by two trained UNMT models through cross-translation, which embraces data diversity. Dataset Previous studies have recommended restricting test sets to natural input sentences, a methodology adopted by the 2019-2020 edition of the WMT news translation shared task (Edunov et al., 2020). In order to further verify the effectiveness of the proposed approach, we also conduct the evaluation on WMT19 and WMT20 En-De test sets. Both test sets contain only natural input samples. Results Experimental results are presented in Table 9. We also show the training costs of these methods. We find that  Unexpectedly, the offline self-training has no significant improvement over baseline UNMT. Sun et al. (2021) have demonstrated the effectiveness of offline self-training in UNMT under low-resource and data imbalanced scenarios. However, in our data-sufficient scenarios, offline self-training may suffer from the data diversity problem while online self-training can alleviate the problem through the dynamic model parameters during the training process. We leave the complete analysis to future work. CBD achieves a significant improvement compared to baseline UNMT, but the training cost is about six times that of online self-training. Since the self-training samples are translated sentences on the target side, there is concern that the improvement achieved by self-training only comes from making the model outputs better match the translated references, rather than enhancing the model's ability on natural inputs. To dispel the concern, we conducted the following experiments: (1) evaluate the fluency of model outputs in terms of language model PPL and (2) evaluate the translation performance on Google Paraphrased WMT19 En  De test sets (Freitag et al., 2020). Output fluency We exploit the monolingual corpora of target languages to train the 4-gram language models. Table 10 shows the language mod-els' PPL on model outputs of test sets mentioned in 5.2. We find that online self-training has only a slight impact on the fluency of model outputs, with the average PPL of XLM and MASS models only increasing by +3 and +6, respectively. We ascribe this phenomenon to the translated target of self-training samples, which is model generated and thus less fluent then natural sentences. However, since the target of BT data is natural and the BT loss term is the primary training objective, the output fluency does not decrease significantly. Translation performance on paraphrased references Freitag et al. (2020) collected additional human translations for newstest2019 with the ultimate aim of generating a natural-to-natural test set. We adopt the HQ(R) and HQ(all 4), which have higher human adequacy rating scores, to re-6617 Approach En-Fr En-De En-Ro Avg. We present the experimental results in Table 11. Our proposed method outperforms baselines on both kinds of test sets. Therefore, we demonstrate that our proposed method improves the UNMT model performance on natural input with limited translationese outputs. Style Gap From Table 8, our proposed approach achieves significant improvements on the natural input portion while not gaining on the translated input portion over the baselines. It indicates our approach has better generalization capability on the natural input portion of test sets than the baselines. Content Gap To verify that our proposed approach bridges the content gap between training and inference, we calculate the accuracy of NER translation by different models. Specifically, we adopt spaCy to recognize the name entities in reference and translation outputs and treat the name entities in reference as the ground truth to calculate the accuracy of NER translation. We show the results in Table 12. Our proposed method achieves a significant improvement in the translation accuracy of NER compared to the baseline. The result demonstrates that online self-training can help the model pay more attention to the input content rather than being affected by the content of the target language training corpus. Next, we investigate the impact of target quality on ST. We use the SNMT model from 2.2 to generate ST data rather than the current model itself and keep the process of BT unchanged. As shown in Table 2, the SNMT models perform well on source-original test set and thus yield higher quality target in ST data. We denote this variant as knowledge distillation (KD) and report the performance on WMT19/20 E  De in Table 13. When target quality gets better, model performance improves significantly, as expected. Therefore, reducing the noise on the target side of the ST data may further improve the performance. Implementing in an unsupervised manner is left to future work. Unsupervised Neural Machine Translation Before attempts to build NMT model using monolingual corpora only, unsupervised cross-lingual embedding mappings had been well studied by Zhang et al. (2017); Artetxe et al. (2017, 2018a); Conneau et al. (2018). These methods try to align the word embedding spaces of two languages without parallel data and thus can be exploited for unsupervised word-by-word translation. Initialized by the cross-lingual word embeddings, Artetxe et al. (2018b) and Lample et al. (2018a) concurrently proposed UNMT, which achieved remarkable performance for the first time using monolingual corpora only. Both of them rely on online back-translation and denoising auto-encoding. After that, Lample et al. (2018b) proposed joint BPE for related languages and combined the neural and phrase-based methods. Artetxe et al. (2019) warmed up the UNMT model by an improved statistical machine translation model. Lample and Conneau (2019) proposed cross-lingual language model pretraining, which obtained large improvements over previous works. Song et al. (2019) extended the pretraining framework to sequence-to-sequence. Tran et al. (2020) induced data diversification in UNMT via cross-model back-translated distillation. Data Augmentation Back-translation (Sennrich et al., 2016; Edunov et al., 2018; Marie et al., 2020) and self-training (Zhang and Zong, 2016; He et al., 2020; Jiao et al., 2021) have been well studied in the supervised NMT. In the unsupervised scenario, Tran et al. (2020) have shown that multilingual pre-trained language models can be used to retrieve the pseudo parallel data from the large monolingual data. Han et al. (2021) use generative pre-training language models, e.g., GPT-3, to perform zero-shot translations and use the translations as few-shot prompts to sample a larger synthetic translations dataset. The most related work to ours is that offline self-training technology used to enhance low-resource UNMT (Sun et al., 2021). In this paper, the proposed online self-training method for UNMT can be applied to both high-resource and low-resource scenarios without extra computation to generate the pseudo parallel data. Translationese Problem Translationese problem has been investigated in machine translation evaluation (Lembersky et al., 2012; Zhang and Toral, 2019; Edunov et al., 2020; Graham et al., 2020). These works aim to analyze the effect of translationese in bidirectional test sets. In this work, we revisit the translationese problem in UNMT and find it causes the inaccuracy evaluation of UNMT performance since the training data entirely comes from the translated pseudo-parallel data. Pseudo parallel corpus generated by back-translation is the foundation of UNMT. However, it also causes the problem of translationese and results in inaccuracy evaluation on UNMT performance. We attribute the problem to the data gap between training and inference and identify two data gaps, i.e., style gap and content gap. We conduct the experiments to evaluate the impact of the data gap on translation performance and propose the online self-training method to alleviate the data gap problems. Our experimental results on multiple language pairs show that the proposed method achieves consistent and significant improvement over the strong baseline XLM and MASS models on the test sets with natural input. Zhiwei He and Rui Wang are with MT-Lab, Department of Computer Science and Engineering, School of Electronic Information and Electrical Engineering, and also with the MoE Key Lab of Arti-ficial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai 200204, China. Rui is supported by General Program of National Natural Science Foundation of China (6217020129), Shanghai Pujiang Program (21PJ1406800), and Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102). Zhiwei is supported by CCF-Tencent Open Fund (RAGR20210119).
abstain abstain abstain method objective result abstain abstain abstain abstain abstain abstain abstain objective abstain abstain result abstain abstain objective abstain abstain abstain method abstain objective abstain objective method objective objective abstain abstain abstain method abstain abstain other abstain abstain abstain method abstain abstain abstain method abstain method method method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method method method abstain abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain other abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method method method method method other abstain abstain abstain result abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other other other other other other other objective other other abstain abstain abstain method objective objective other other other