The principal task in supervised neural machine translation (NMT) is to learn to generate target sentences conditioned on the source inputs from a set of parallel sentence pairs, and thus produce a model capable of generalizing to unseen instances. However, it is commonly observed that the generalization performance of the model is highly influenced by the amount of parallel data used in training. Although data augmentation is widely used to enrich the training data, conventional methods with discrete manipulations fail to generate diverse and faithful training samples. In this paper, we present a novel data augmentation paradigm termed Continuous Semantic Augmentation (CSANMT), which augments each training instance with an adjacency semantic region that could cover adequate variants of literal expression under the same meaning. We conduct extensive experiments on both rich-resource and low-resource settings involving various language pairs, including WMT14 English  {German,French}, NIST Chinese  English and multiple low-resource IWSLT translation tasks. The provided empirical evidences show that CSANMT sets a new level of performance among existing augmentation techniques, improving on the state-of-the-art by a large margin. 1 1 Introduction Neural machine translation (NMT) is one of the core topics in natural language processing, which aims to generate sequences of words in the target language conditioned on the source inputs (Sutskever et al., 2014; Cho et al., 2014; Wu et al., 2016; Vaswani et al., 2017). In the common supervised setting, the training objective is to learn a transformation from the source space to the target space X (cid:55) Y : f ( y | x ; ) with the usage of parallel data. In this way, NMT models are expected to 1 The core codes are contained in Appendix E. be capable of generalizing to unseen instances with the help of large scale training data, which poses a big challenge for scenarios with limited resources. To address this problem, various methods have been developed to leverage abundant unlabeled data for augmenting limited labeled data (Sen-nrich et al., 2016a; Cheng et al., 2016; He et al., 2016; Hoang et al., 2018; Edunov et al., 2018; He et al., 2020; Song et al., 2019). For example, back-translation (BT) (Sennrich et al., 2016a) makes use of the monolingual data on the target side to synthesize large scale pseudo parallel data, which is further combined with the real parallel corpus in machine translation task. Another line of research is to introduce adversarial inputs to improve the generalization of NMT models towards small perturbations (Iyyer et al., 2015; Fadaee et al., 2017; Wang et al., 2018; Cheng et al., 2018; Gao et al., 2019). While these methods lead to significant boosts in translation quality, we argue that augmenting the observed training data in the discrete space inherently has two major limitations. First, augmented training instances in discrete space are lack diversity. We still take BT as an example, it typically uses beam search (Sennrich et al., 2016a) or greedy search (Lample et al., 2018a,c) to generate synthetic source sentences for each target monolingual sentence. The above two search strategies are approximate algorithms to identify the maximum a-posteriori (MAP) output (Edunov et al., 2018), and thus favor the most frequent one in case of ambiguity. Edunov et al. (2018) proposed a sampling strategy from the output distribution to alleviate this issue, but this method typically yields synthesized data with low quality. While some extensions (Wang et al., 2018; Imamura et al., 2018; Khayrallah et al., 2020; Nguyen et al., 2020) augment each training instance with multiple literal forms, they still fail to cover adequate variants under the same meaning. crete space to preserve their original meanings. In the context of natural language processing, discrete manipulations such as adds, drops, reorders, and/or replaces words in the original sentences often result in significant changes in semantics. To address this issue, Gao et al. (2019) and Cheng et al. (2020) instead replace words with other words that are predicted using language model under the same context, by interpolating their embeddings. Although being effective, these techniques are limited to word-level manipulation and are unable to perform the whole sentence transformation, such as producing another sentence by rephrasing the original one so that they have the same meaning. In this paper, we propose C ontinuous S emantic A ugmentation (CSANMT), a novel data augmentation paradigm for NMT, to alleviate both limitations mentioned above. The principle of CSANMT is to produce diverse training data from a semantically-preserved continuous space. Specifically, (1) we first train a semantic encoder via a tangential contrast, which encourages each training instance to support an adjacency semantic region in continuous space and treats the tangent points of the region as the critical states of semantic equivalence. This is motivated by the intriguing observation made by recent work showing that the vectors in continuous space can easily cover adequate variants under the same meaning (Wei et al., 2020a). (2) We then introduce a Mixed Gaussian Recurrent Chain (MGRC ) algorithm to sample a cluster of vectors from the adjacency semantic region. (3) Each of the sampled vectors is finally incorporated into the decoder by developing a broadcasting integration network , which is agnostic to model architectures. As a consequence, transforming discrete sentences into the continuous space can effectively augment the training data space and thus improve the generalization capability of NMT models. We evaluate our framework on a variety of machine translation tasks, including WMT14 English-German/French, NIST Chinese-English and multiple IWSLT tasks. Specifically, CSANMT sets the new state of the art among existing augmentation techniques on the WMT14 English-German task with 30.94 BLEU score. In addition, our approach could achieve comparable performance with the baseline model with the usage of only 25% of training data. This reveals that CSANMT has great potential to achieve good results with very few data. Furthermore, CSANMT demonstrates consistent improvements over strong baselines in low resource scenarios, such as IWSLT14 English-German and IWSLT17 English-French. Problem Definition Supposing X and Y are two data spaces that cover all possible sequences of words in source and target languages, respectively. We denote ( x , y )  ( X , Y ) as a pair of two sentences with the same meaning, where x = { x 1 , x 2 , ..., x T } is the source sentence with T tokens, and y = { y 1 , y 2 , ..., y T  } is the target sentence with T  tokens. A sequence-to-sequence model is usually applied to neural machine translation, which aims to learn a transformation from the source space to the target space X (cid:55) Y : f ( y | x ; ) with the usage of parallel data. Formally, given a set of observed sentence pairs C = { ( x ( n ) , y ( n ) ) } N n =1 , the training objective is to maximize the log-likelihood: J mle () = E ( x , y ) C (cid:0) log P ( y | x ; ) (cid:1) . The log-probability is typically decomposed as: log P ( y | x ; ) = (cid:80) T  t =1 log P ( y t | y <t , x ; ) , where  is a set of trainable parameters and y <t is a partial sequence before time-step t . However, there is a major problem in the common supervised setting for neural machine translation, that is the number of training instances is very limited because of the cost in acquiring parallel data. This makes it difficult to learn an NMT model generalized well to unseen instances. Traditional data augmentation methods generate more training samples by applying discrete manipulations to unlabeled (or labeled) data, such as back-translation or randomly replacing a word with another one, which usually suffer from the problems of semantic deviation and the lack of diversity. We propose a novel data augmentation paradigm for neural machine translation, termed continuous semantic augmentation (CSANMT), to better generalize the model's capability to unseen instances. We adopt the Transformer (Vaswani et al., 2017) model as a backbone, and the framework is shown in Figure 1. In this architecture, an extra semantic encoder translates the source x and the target sentence y to real-value vectors r x =  ( x ;   ) and r y =  ( y ;   ) respectively, where  (  ;   ) is the forward function of the semantic encoder parameterized by   (parameters other than  ). Definition 1. There is a universal semantic space among the source and the target languages for neural machine translation, which is established by a semantic encoder. It defines a forward function  (  ;   ) to map discrete sentences into continuous vectors, that satisfies:  ( x , y )  ( X , Y ) : r x = r y . Besides, an adjacency semantic region  ( r x , r y ) in the semantic space describes adequate variants of literal expression centered around each observed sentence pair ( x , y ) . In our scenario, we first sample a series of vectors (denoted by R ) from the adjacency semantic region to augment the current training instance, that is R = {  r (1) ,  r (2) , ...,  r ( K ) } , where  r ( k )   ( r x , r y ) . K is the hyperparameter that determines the number of sampled vectors. Each sample  r ( k ) is then integrated into the generation process through a broadcasting integration network:  o t = W 1  r ( k ) + W 2 o t + b, (2) where o t is the output of the self-attention module at position t . Finally, the training objective in Eq. (1) can be improved as J mle () = E ( x , y ) C ,  r ( k ) R (cid:0) log P ( y | x ,  r ( k ) ; ) (cid:1)(cid:1) . By augmenting the training instance ( x , y ) with diverse samples from the adjacency semantic region, the model is expected to generalize to more unseen instances. To this end, we must consider such two problems: (1) How to optimize the semantic encoder so that it produces a meaningful adjacency semantic region for each observed training pair. (2) How to obtain samples from the adjacency semantic region in an efficient and effective way. In the rest part of this section, we introduce the resolutions of these two problems, respectively. Tangential Contrastive Learning We start from analyzing the geometric interpretation of adjacency semantic regions. The schematic diagram is illustrated in Figure 2. Let ( x ( i ) , y ( i ) ) and ( x ( j ) , y ( j ) ) are two instances randomly sampled from the training corpora. For ( x ( i ) , y ( i ) ) , the adjacency semantic region  ( r x ( i ) , r y ( i ) ) is defined as the union of two closed balls that are centered by r x ( i ) and r y ( i ) , respectively. The radius of both balls is d =  r x ( i )  r y ( i )  2 , which is also considered as a slack variable for determining semantic equivalence. The underlying interpretation is that vectors whose distances from r x ( i ) (or r y ( i ) ) do not exceed d , are semantically-equivalent to both r x ( i ) and r y ( i ) . To make  ( r x ( i ) , r y ( i ) ) conform to the interpretation, we employ a similar method as in (Zheng et al., 2019; Wei et al., 2021) to optimize the semantic encoder with the tangential contrast. Specifically, we construct negative samples by applying the convex interpolation between the current instance and other ones in the same training batch for instance comparison. And the tangent points (i.e., the points on the boundary) are considered as the critical states of semantic equivalence. The training objective is formulated as: J ctl (  ) = E ( x ( i ) , y ( i ) ) B (cid:18) log e s (cid:0) r x ( i ) ,r y ( i ) (cid:1) e s (cid:0) r x ( i ) ,r y ( i ) (cid:1) +  (cid:19) ,  = |B| (cid:88) j & j  = i (cid:16) e s (cid:0) r y ( i ) ,r y  ( j ) (cid:1) + e s (cid:0) r x ( i ) ,r x  ( j ) (cid:1)(cid:17) , (4) where B indicates a batch of sentence pairs randomly selected from the training corpora C , and s (  ) is the score function that computes the cosine similarity between two vectors. The negative samples r x  ( j ) and r y  ( j ) are designed as the following 7932 Figure 3: The geometric diagram of the proposed MGRC sampling. r x  ( j ) = r x ( i ) +  x ( r x ( j )  r x ( i ) ) ,  x  ( d d  x , 1] , r y  ( j ) = r y ( i ) +  y ( r y ( j )  r y ( i ) ) ,  y  ( d d  y , 1] , (5) where d  x =  r x ( i )  r x ( j )  and d  y =  r y ( i )  r y ( j )  . The two equations in Eq. (5) set up when d  x and d  y are larger than d respectively, or else r x  ( j ) = r x ( j ) and r y  ( j ) = r y ( j ) . According to this design, an adjacency semantic region for the i -th training instance can be fully established by interpolating various instances in the same training batch. We follow Wei et al. (2021) to adaptively adjust the value of  x (or  y ) during the training process, and refer to the original paper for details. MGRC Sampling To obtain augmented data from the adjacency semantic region for the training instance ( x , y ) , we introduce a Mixed Gaussian Recurrent Chain (denoted by MGRC ) algorithm to design an efficient and effective sampling strategy. As illustrated in Figure 3, we first transform the bias vector  r = r y  r x according to a predefined scale vector  , that is    r , where  is the element-wise product operation. Then, we construct a novel sample  r = r +    r for augmenting the current instance, in which r is either r x or r y . As a consequence, the goal of the sampling strategy turns into find a set of scale vectors, i.e. {  (1) ,  (2) , ...,  ( K ) } . Intuitively, we can assume that  follows a distribution with universal or Gaussian forms, despite the latter demonstrates better results in our experience. Formally, we design a Algorithm 1 MGRC Sampling Input: The representations of the training instance ( x , y ) , i.e. r x and r y . mixed Gaussian distribution as follow: ( k )  p (  |  (1) ,  (2) , ...,  ( k  1) ) , p =  N (cid:0) 0 , diag( W 2 r ) (cid:1) + (1 . 0   ) N (cid:18) 1 k  1 k  1 (cid:88) i =1  ( i ) , 1 (cid:19) . (6) This framework unifies the recurrent chain and the rejection sampling mechanism. Concretely, we first normalize the importance of each dimension in  r as W r = |  r | min ( |  r | ) max ( |  r | )  min ( |  r | ) , the operation |  | takes the absolute value of each element in the vector, which means the larger the value of an element is the more informative it is. Thus N ( 0 , diag( W 2 r )) limits the range of sampling to a subspace of the adjacency semantic region, and rejects to conduct sampling from the uninformative dimensions. Moreover, N ( 1 k  1 (cid:80) k  1 i =1  ( i ) , 1 ) simulates a recurrent chain that generates a sequence of reasonable vectors where the current one is dependent on the prior vectors. The reason for this design is that we expect that p in Eq. (6) can become a stationary distribution with the increase of the number of samples, which describes the fact that the diversity of each training instance is not infinite. is a hyperparameter to balance the importance of the above two Gaussian forms. For a clearer presentation, Algorithm 1 summarizes the sampling process. The training objective in our approach is a combination of J mle () in Eq. (3) and J ctl (  ) in Eq. (4). In practice, we introduce a two-phase training procedure with mini-batch losses. Firstly, we train the semantic encoder from scratch using the task-specific data, i.e.   = argmax   J ctl (  ) . Secondly, we optimize the encoder-decoder model by maximizing the log-likelihood, i.e.   = argmax  J mle () , and fine-tune the semantic encoder with a small learning rate at the same time. During inference, the sequence of target words is generated auto-regressively, which is almost the same as the vanilla Transformer (Vaswani et al., 2017). A major difference is that our method involves the semantic vector of the input sequence for generation: y  t = argmax y t P ( | y <t , x , r x ; ) , where r x =  ( x ;   ) . This module is plug-in-use as well as is agnostic to model architectures. We first apply CSANMT to NIST Chinese-English (Zh  En), WMT14 English-German (En  De) and English-French (En  Fr) tasks, and conduct extensive analyses for better understanding the proposed method. And then we generalize the capability of our method to low-resource IWSLT tasks. Datasets. For the Zh  En task, the LDC corpus is taken into consideration, which consists of 1.25M sentence pairs with 27.9M Chinese words and 34.5M English words, respectively. The NIST 2006 dataset is used as the validation set for selecting the best model, and NIST 2002 (MT02), 2003 (MT03), 2004 (MT04), 2005 (MT05), 2008 (MT08) are used as the test sets. For the En  De task, we employ the popular WMT14 dataset, which consists of approximately 4.5M sentence pairs for training. We select newstest2013 as the validation set and newstest2014 as the test set. For the En  Fr task, we use the significantly larger WMT14 dataset consisting of 36M sentence pairs. The combination of { newstest2012, 2013 } was used for model selection and the experimental results were reported on newstest2014 . Refer to Appendix A for more details. Training Details. We implement our approach on top of the Transformer (Vaswani et al., 2017). The semantic encoder is a 4-layer transformer encoder with the same hidden size as the backbone model. Following sentence-bert (Reimers and Gurevych, 2019), we average the outputs of all positions as the sequence-level representation. The learning rate for finetuning the semantic encoder at the second training stage is set as 1 e  5 . All experiments are performed on 8 V100 GPUs. We accumulate the gradient of 8 iterations and update the models with a batch of about 65K tokens. The hyperparameters K and  in MGRC sampling are tuned on the validation set with the range of K  { 10 , 20 , 40 , 80 } and   { 0 . 15 , 0 . 30 , 0 . 45 , 0 . 6 , 0 . 75 , 0 . 90 } . We use the default setup of K = 40 for all three tasks,  = 0 . 6 for both Zh  En and En  De while  = 0 . 45 for En  Fr. For evaluation, the beam size and length penalty are set to 4 and 0.6 for the En  De as well as En  Fr, while 5 and 1.0 for the Zh  En task. Results of Zh  En. Table 1 shows the results on the Chinese-to-English translation task. From the results, we can conclude that our approach outperforms existing augmentation strategies such as back-translation (Sennrich et al., 2016a; Wei et al., 2020a) and switchout (Wang et al., 2018) by a large margin (up to 3.63 BLEU), which verifies that augmentation in continuous space is more effective than methods with discrete manipulations. Compared to the approaches that replace words in the embedding space (Cheng et al., 2020), our approach also demonstrates superior performance, which reveals that sentence-level augmentation with continuous semantics works better on generalizing to unseen instances. Moreover, compared to the vanilla Transformer, our approach consistently 7934 Model WMT 2014 En  De WMT 2014 En  Fr #Params. achieves promising improvements on five test sets. Results of En  De and En  Fr. From Table 2, our approach consistently performs better than existing methods (Sennrich et al., 2016a; Wang et al., 2018; Wei et al., 2020a; Cheng et al., 2020), yielding significant gains (0.65  1.76 BLEU) on the En  De and En  Fr tasks. An exception is that Nguyen et al. (2020) achieved comparable results with ours via multiple forward and backward NMT models, thus data diversification intuitively demonstrates lower training efficiency. Moreover, we observe that CSANMT gives 30.16 BLEU on the En  De task with the base setting, significantly outperforming the vanilla Transformer by 2.49 BLEU points. Our approach yields a further improvement of 0.68 BLEU by equipped with the wider architecture, demonstrating superiority over the standard Transformer by 2.15 BLEU. Similar observations can be drawn for the En  Fr task. Effects of K and  . Figure 4 illustrates how the hyper-parameters K and  in MGRC sampling affect the translation quality. From Figures 4(a)-4(c), we can observe that gradually increasing the number of samples significantly improves BLEU scores, which demonstrates large gaps between K = 10 and K = 40 . However, assigning larger values (e.g., 80 ) to K does not result in further improvements among all three tasks. We conjecture that the reasons are two folds: (1) it is fact that the diversity of each training instance is not infinite and thus MGRC gets saturated is inevitable with K increasing. (2) MGRC sampling with a scaled item (i.e., W r ) may degenerate to traverse in the same place. This prompts us to design more sophisticated algorithms in future work. In our experiments, we default set K = 40 to achieve a balance between the training efficiency and translation quality. Figure 4(d) shows the effect of  on validation sets, which balances the importance of two Gaussian forms during the sampling process. The setting of  = 0 . 6 achieves the best results on both the Zh  En and En  De tasks, and  = 0 . 45 consistently outperforms other values on the En  Fr task. Lexical diversity and semantic faithfulness. We demonstrate both the lexical diversity (mea-sured by TTR = num. of types num. of tokens ) of various trans-7935 0.00 0.25 0.50 0.75 1.00 Ratio of the Training Data 16 20 24 28 32 BLEU ( % ) Ours Back-translation + Mono. lations and the semantic faithfulness of machine translated ones (measured by BLEURT with considering human translations as the references) in Table 4. It is clear that CSANMT substantially bridge the gap of the lexical diversity between translations produced by human and machine. Meanwhile, CSANMT shows a better capability on preserving the semantics of the generated translations than Transformer. We intuitively attribute the significantly increases of BLEU scores on all datasets to these two factors. We also have studied the robustness of CSANMT towards noisy inputs and the translationese effect, see Appendix D for details. Effect of the semantic encoder. We introduce two variants of the semantic encoder to investigate its performance on En  De validation set. Specifically, (1) we remove the extra semantic encoder and construct the sentence-level representations by averaging the sequence of outputs of the vanilla sentence encoder. (2) We replace the default 4-layer semantic encoder with a large pre-trained model (PTM) (i.e., XLM-R (Conneau et al., 2020)). The results are reported in Table 3. Comparing line 2 with line 3, we can conclude that an extra semantic encoder is necessary for constructing the universal continuous space among different languages. Moreover, when the large PTM is incorporated, our approach yields further improvements, but it causes massive computational overhead. isons between different augmentation methods, we asymptotically increase the training data to analyze the performance of them on the En  De translation. As in Figure 5, our approach significantly outperforms the back-translation method on each subset, whether or not extra monolingual data (Sen-nrich et al., 2016a) is introduced. These results demonstrate the stronger ability of our approach than discrete augmentation methods on generalizing to unseen instances with the same set of observed data points. Encouragingly, our approach achieves comparable performance with the baseline model with only 25% of training data, which indicates that our approach has great potential to achieve good results with very few data. Effect of MGRC sampling and tangential contrastive learning. To better understand the effectiveness of the MGRC sampling and the tangential contrastive learning, we conduct detailed ablation studies in Table 5. The details of four variants with different objectives or sampling strategies are shown in Appendix C . From the results, we can observe that both removing the recurrent dependence and replacing the Gaussian forms with uniform distributions make the translation quality decline, but the former demonstrates more drops. We also have tried the training objectives with other forms, such as variational inference and cosine similarity, to optimize the semantic encoder. However, the BLEU score drops significantly. shows the evolution of BLEU scores during training. It is obvious that our method performs consistently better than both the vanilla Transformer and the back-translation method at each iteration (ex-cept for the first 10K warm-up iterations, where the former one has access to less unique training data than the latter two due to the K times over-sampling). For the vanilla Transformer, the BLEU score reaches its peak at about 52K iterations. In comparison, both CSANMT and the back-translation method require 75K updates for convergence. In other words, CSANMT spends 44% more training costs than the vanilla Transformer, due to the longer time to make the NMT model converge with augmented training instances. This is the same as the back-translation method. Word prediction accuracy. Figure 7 illustrates the prediction accuracy of both frequent and rare words. As expected, CSANMT generalizes to rare words better than the vanilla Transformer, and the gap of word prediction accuracy is as large as 16%. This indicates that the NMT model alleviates the probability under-estimation of rare words via continuous semantic augmentation. Baselines. In contrast to the vanilla Transformer, CSANMT involves with approximate 20% additional parameters. In this section, we further compare against the baselines with increased amounts of parameters, and investigate the performance of CSANMT equipped with much stronger baselines (e.g. deep and scale Transformers (Ott et al., 2018; Wang et al., 2019; Wei et al., 2020b)). From the results on WMT14 testsets in Table 6, we can observe that CSANMT still outperforms the vanilla Transformer (by more than 1.2 BLEU) under the same amount of parameters, which shows that the additional parameters are not the key to the improvement. Moreover, CSANMT yields at least 0.9 BLEU gains equipped with much stronger baselines. For example, the scale Transformer (Ott et al., 2018), which originally gives 29.3 BLEU in the En  De task, now gives 31.37 BLEU with our continuous semantic augmentation strategy. It is important to mention that our method can help models to achieve further improvement, even if they are strong enough. We further generalize the capability of the proposed CSANMT to various low-resource machine translation tasks, including IWSLT14 English-German and IWSLT17 English-French. The details of the datasets and model configurations can be found in Appendix B . Table 7 shows the results of different models. Compared to the vanilla Transformer, the proposed CSANMT improve the BLEU scores of the two tasks by 2.7 and 2.9 points, respectively. This result indicates that the claiming of the continuous semantic augmentation enriching the training corpora with very limited observed instances. Data Augmentation (DA) (Edunov et al., 2018; Kobayashi, 2018; Gao et al., 2019; Khayrallah et al., 2020; Pham et al., 2021) has been widely used in neural machine translation. The most popular one is the family of back-translation (Sennrich et al., 2016a; Nguyen et al., 2020), which utilizes a target-to-source model to translate monolingual target sentences back into the source language. Besides, constructing adversarial training instances with diverse literal forms via word replacing or embedding interpolating (Wang et al., 2018; Cheng et al., 2020) is beneficial to improve the generalization performance of NMT models. Vicinal Risk Minimization (VRM) (Chapelle et al., 2000) is another principle of data augmentation, in which DA is formalized as extracting additional pseudo samples from the vicinal distribution of observed instances. Typically the vicinity of a training example is defined using dataset-dependent heuristics, such as color (scale, mixup) augmentation (Simonyan and Zisserman, 2014; Krizhevsky et al., 2012; Zhang et al., 2018) in computer vision and adversarial augmentation with manifold neighborhoods (Ng et al., 2020; Cheng et al., 2021) in NLP. Our approach relates to VRM that involves with an adjacency semantic region as the vicinity manifold for each training instance. Sentence Representation Learning is a well investigated area with dozens of methods (Kiros et al., 2015; Cer et al., 2018; Yang et al., 2018). In recent years, the methods built on large pre-trained models (Devlin et al., 2019; Conneau et al., 2020) have been widely used for learning sentence level representations (Reimers and Gurevych, 2019; Huang et al., 2019; Yang et al., 2019). Our work is also related to the methods that aims at learning the universal representation (Zhang et al., 2016; Schwenk and Douze, 2017; Yang et al., 2021) for multiple semantically-equivalent sentences in NMT. In this context, contrastive learning has become a popular paradigm in NLP (Kong et al., 2020; Clark et al., 2020; Gao et al., 2021). The most related work are Wei et al. (2021) and Chi et al. (2021), which suggested transforming cross-lingual sentences into a shared vector by contrastive objectives. We propose a novel data augmentation paradigm CSANMT, which involves with an adjacency semantic region as the vicinity manifold for each training instance. This method is expected to make more unseen instances under generalization with very limited training data. The main components of CSANMT consists of the tangential contrastive learning and the Mixed Gaussian Recurrent Chain (MGRC ) sampling. Experiments on both richand low-resource machine translation tasks demonstrate the effectiveness of our method. In the future work, we would like to further study the vicinal risk minimization with the combination of multi-lingual aligned scenarios and large-scale monolingual data, and development it as a pure data augmentator merged into the vanilla Transformer. We would like to thank all of the anonymous reviewers (during ARR Oct. and ARR Dec.) for the helpful comments. We also thank Baosong Yang and Dayiheng Liu for their instructive suggestions and invaluable help.
abstain abstain abstain objective method abstain abstain abstain abstain abstain abstain abstain objective abstain method abstain abstain abstain abstain abstain abstain abstain objective abstain objective abstain method abstain abstain method abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain other abstain other abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain other other other other other abstain other other abstain other other objective abstain abstain objective method other other