Shujian Huang Shuming Shi Jiajun Chen 1 National Key Laboratory for Novel Software Technology, Nanjing University, China Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 2 Abstract Interactive neural machine translation (INMT) is able to guarantee high-quality translations by taking human interactions into account. Existing IMT systems relying on lexical constrained decoding (LCD) enable humans to translate in a flexible translation manner beyond left-to-right. However, they typically suffer from limitations in translation efficiency and quality due to the reliance on LCD. In this work, we propose a novel BiTIIMT system, Bi lingual T extI nfilling for I nteractive Neural M achine T ranslation. The key idea to BiTIIMT is the Bilingual Text-infilling (BiTI) task which aims to fill missing segments in a manually revised translation for a given source sentence. We propose a simple yet effective solution by casting this task as a sequence-to-sequence task. The benefits of our solution are that it performs efficient decoding with the same complexity as the standard decoding in NMT and makes full use of revised words for better translation prediction. Experiment results show that BiTiIMT performs significantly better and faster than state-of-the-art LCD-based IMT on three translation tasks. Recent years have witnessed significant development in neural machine translation(NMT) (Bah-danau et al., 2015; Vaswani et al., 2017). Despite their success, their translation in quality still can not meet the requirements in industrial applications. On the other hand, interactive neural machine translation (IMT) (Foster et al., 1997; Langlais et al., 2000; Simard et al., 2007; Barrachina et al., 2009; Gonzlez-Rubio et al., 2013; Cheng et al., 2016; Weng et al., 2019; Huang et al., 2021) is able to guarantee high-quality translation: it is an iterative collaboration process between human and machine that involves multiple interactive steps to obtain a satisfactory translation. Work was done during internship at Tencent AI Lab. Corresponding authors. Traditional IMT generates a translation in the left-to-right completing paradigm (Langlais et al., 2000; Sanchis-Trilles et al., 2014; Peris et al., 2017a; Knowles and Koehn, 2016; Zhao et al., 2020) where human translators are required to revise words in the translation prefix. This strict left-to-right manner limits its flexibility because some human translators may enjoy their translation man-ners beyond left-to-right. As a result, another part of works (Weng et al., 2019; Huang et al., 2021) propose an alternative IMT paradigm under which human translators can revise words at arbitrary positions of a translation. The essential technique to this paradigm is lexically constrained decoding (LCD) (Hokamp and Liu, 2017; Post and Vilar, 2018), which extends beam search in the decoding stage to include revised words as constraints. Unfortunately, LCD-based IMT suffers from two major shortcomings on efficiency and translation quality in practice. Firstly, LCD-based IMT usually involves multiple interactions between a human translator and machine and runs the LCD algorithm multiple times to translate a sentence. Since each LCD run takes considerable time compared with NMT decoding, the human translator will encounter severe latency, leading to poor user experience. In addition, LCD is based on the standard translation model, which is defined on top of the prefix context, and thus cannot make use of the revised words to assist the model in predicting target words to their left. Hence this characteristic limits its overall translation quality. This paper proposes a simple yet effective IMT approach, BiTIIMT, which addresses the issues above. The core idea to BiTIIMT is the Bilingual Text-infilling (BiTI) task which extends text-infilling (Zhu et al., 2019) from monolingual setting to bilingual setting and aims to fill missing segments in a revised translation for a given source sentence. Unlike Zhu et al. (2019) carefully designing a model, we simply cast the bilingual text-infilling 1958 task as a sequence-to-sequence task and then employ the standard NMT model to perform this task. To train the model, we construct simulated data by randomly sampling revised sentences from a bilingual corpus and augment the simulated data with bilingual corpus for further improvements. In this way, our model is able to yield a valid output that can be seamlessly filled in the revised translation in an efficient way similar to the standard NMT decoding. Moreover, the proposed model makes full use of all revised words to predict a target word and thus has the potential to obtain better translation than LCD. We conduct extensive experiments on WMT14 En-De, WMT14 En-Fr, and Zh-En tasks. Our simulated experiments demonstrate that the proposed model indeed outperforms LCD in terms of translation quality and efficiency, and the proposed BiTIIMT is better than LCDIMT according to both translation quality and human editing costs. The advantages of BiTIIMT over LCDIMT are also ver-ified in real-world IMT experiments with human translators. This paper makes the following contributions:  It proposes the bilingual text-infilling task and provides a simple yet effective solution to address this task. It proposes a novel IMT system on top of bilingual text-infilling which empirically outperforms a strong baseline in both translation quality and efficiency. Neural machine translation (NMT) (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017) is based on a sequence to sequence model which adopts an encoder-decoder architecture. The encoder summarizes the source sentence into an intermediate representation, and the decoder generates the target sentence. Given a source sentence X = { x 1 ,    , x i ,    } , a NMT model factorizes the distribution over possible output Y = { y 1 ,    , y T } into a chain of conditional probabilities from left to right: P ( Y | X ;  ) = T +1 (cid:89) t =1 P ( y t | y 1 , y 2 ,    , y t  1 , X ;  ) (1) Source , Text infilling (Zhu et al., 2019) is a task that fills missing text segments of a sentence or paragraph by a model (Berglund and Leo; Fedus et al., 2018; Zhu et al., 2019) trained on a large amount of data in a fill-in-the-blank format. The input text X = { x 1 , <blank> , x i , ..., <blank> , ... } has an unknown number of blanks whose positions are arbitrary, and each blank has an arbitrary unknown length. To address this task, a text infilling model fills in each blank from left to right by predicting a target word y j at each time step j . As a solution, Zhu et al. (2019) proposes a variant model based on Transformer, whose position encoding takes both segment positions and token positions into account. This section illustrates the overview of the proposed BiTIIMT system and accordingly presents its essential technique (i.e., bilingual text-infilling) to take human interactions into NMT. In general, the proposed BiTIIMT enjoys a human-in-the-loop manner to output the final translation, similar to conventional IMT systems (Cheng et al., 2016). Specifically, for a given source sentence X , BiTIIMT iteratively performs the following two steps:  A human translator edits a translation Y from the translation engine; 1959  Then the engine updates Y based on the edited translation as well as its source X . This procedure terminates until the human translator is satisfied with the quality of Y . This procedure is illustrated in Figure 1. The key to BiTIIMT is its second step which relies on Bilingual Text-infilling to update a translation Y . In the rest of this section, the details about bilingual text-infilling will be described. Problem Statement Generally, bilingual text-infilling extends text-infilling from the monolingual setting (Zhu et al., 2019) to the bilingual setting. Suppose  Y is a template, i.e., the edited (partial) translation, which includes some blanks to be filled in; Y b = { y b 1 , y b 2 ,    , } is a sequence of segments used to fill the blanks in  Y . BiTI aims to generate Y b for filling the blanks in  Y , to obtain a translation Y for a source sentence X . Take Figure 2 as an example, the template  Y is a partial translation edited by a translator which contains three blanks. Y b includes three segments to fill each blank in  Y , and Y is the translation after filling  Y with Y b . It is worth noting that Y b contains three special tokens <eob>, indicating the end of a segment, which correspond to the blanks in  Y , respectively. It is an inherent right __ all measures __ preserve __ ! :  ! : that must be upheld by all Member States, and <eob> must be taken to <eob> it. <eob> It is an inherent right that must be upheld by all Member States, and all measures must be taken to preserve it. : Figure 2: Main notations of bilingual text-infilling where represent a blank based on example in Figure 1. , <sep> It is an inherent right __ all measures __ preserve __ <sep>  <sep> " :  ! : that must be upheld by all Member States, and <eob> must be taken to <eob> it. <eob> Figure 3: Bilingual text-infilling as the sequence-to-sequence task. To implement this model, it is possible to extend the Transformer (Vaswani et al., 2017) by using two encoders (i.e., one is for X , and the other is for  Y ) and taking both segment and token positions into account similar to Zhu et al. (2019). However, for simplicity, we instead cast this task as a standard sequence-to-sequence task by format manipulation and employ an NMT model to address it. Specifically, we treat two input sequences X and  Y as one input sequence  X <sep>  Y , where <sep> is a speck token for concatenation, and Y b as the output sequence, as shown in Figure 3. Then we employ the Transformer model to accomplish this task as the conventional NMT task. Relation to Previous Work Our method is similar to previous works, including lexically constrained decoding (LCD) (Hokamp and Liu, 2017; Post and Vilar, 2018), MT with soft constraints (Dinu et al., 2019) and code-switch enhanced MT (Song et al., 2019; Chen et al., 2020) in the sense that all of them generate translations based on given constraints which are  Y in our work. However, LCD imposes hard constraints on beam search during the decoding stage, leading to the suffering of decoding speed. In addition, the other two series of works can not guarantee the constraints  Y will be in the output translation, and the code-switch method even requires word alignment information whereas human translators do not provide such alignment information in our scenario. Typically, to train the model in Eq.(2), one needs to obtain large amount of data consisting of triples D = {(cid:104) X,  Y , Y b (cid:105)} . Unfortunately, this is impractical because both  Y and Y b are obtained by human translators. To this end, we apply a simple simulation method to obtain {(cid:104) X,  Y , Y b (cid:105)} on top of bilingual corpus {(cid:104) X, Y (cid:105)} through random sampling. Specifically, for each oracle target sentence Y in bilingual data, we randomly sample an integer k  [0 , 5] and randomly sample k non-overlapped segments in Y . Then  Y can be obtained by replacing each remaining segment with <blank> in Y , and Y b can be obtained by wrapping k sampled segments and then joining them together in the same order. This sampling procedure is shown in Figure 4. Given a training data D = {(cid:104) X,  Y , Y b (cid:105)} , it is straightforward to optimize the following objective function according to maximum likelihood estimation: (cid:96) =  (cid:88) (cid:104) X,  Y ,Y b (cid:105) log P ( Y b | X,  Y ;  ) where the model P is defined in Eq. (2). Models trained on training data D = {(cid:104) X,  Y , Y b (cid:105)} possess the ability to translate sentences with constraints in fill-in-blank format style, but it may lose its strength in translating normal sentences. As a result, for the first iteration in IMT, the initial translation maybe not good as expected, leading to more interactive iterations between humans and machines. One possible solution is to build an additional NMT model to generate the initial translation. Instead, we apply the data augmentation (DA) technique, making our model perform both tasks. For the standard bilingual parallel data {(cid:104) X, Y (cid:105)} , we construct its trivial triple data D (cid:48) = {(cid:104) X,  , Y (cid:105)} , where  means that no revised words are provided in  Y for each bilingual sentence. Then we combine the sampled data D and the trivial data D (cid:48) as the augmented data to train the model P in our experiments. Actually, our method is slightly different from classical data augmentation for NMT (Novak et al., 2018; Wang et al., 2018; Li et al., 2019) because the input in the augmented data is different from that for bilingual text infilling task. In addition, because the augmented data is used for bilingual text infilling task and the other data is used for translation task, our training method resembles an instance of multi-task learning (MTL) framework(Caruana, 1997; Dong et al., 2015; Liu et al., 2016; Wang et al., 2020b), where both tasks are modeled by the same Transformer architecture. The task of bilingual text-infilling is reduced to decode Y b according to the model P in Eq. (2) via maximum a posteriori (MAP) estimation as follows: arg max Y b :# b ( Y b )=# b ( Y ) P ( Y b | X,  Y ;  ) (3) where # b denotes the number of blanks, i.e., # b ( Y b ) is the number of <eob> in Y b and # b (  Y ) counts the number of   in  Y . The constraints in the above equation are used to guarantee that all blanks in  Y can be exactly filled by Y b to obtain a valid translation Y , otherwise, Y b would lead to an invalid Y . Theoretically, the constrained optimization in Eq. (3) is more difficult than the unconstrained one in standard NMT decoding. In practice, since the constraint in Eq. (3) is about the number of blanks in Y b , it is easy to satisfy by extending the standard beam search algorithm. Specifically, in the standard beam search algorithm, one only needs to maintain a number to restore the number of blanks in the partial output Y b . If this number is equal to # b (  Y ) , Y b is the final output; otherwise, Y b should be extended until the constraint is satisfied. As a result, our decoding algorithm is very efficient, and it shares the same complexity as the standard beam search algorithm. In fact, thanks to the powerful Transformer architecture and sufficient training data in our scenario, our model is able to implicitly learn the constraint # b ( Y b ) = # b (  Y ) with about 99.39% accuracy in our preliminary experiments. In other words, for almost all sentences, the standard beam search algorithm is able to yield a valid Y , even without explicitly imposing the constraint during decoding. Following previous works (Peris et al., 2017b; Cheng et al., 2016; Weng et al., 2019; Li et al., 2020), we experiment on two simulated scenarios and a real-world scenario. We conduct experiments on the English-German dataset (En-De), English-French (En-Fr), and a Chinese-English dataset (Zh-En), which includes about 2 million bilingual sentences from the news domain in total. For En-De and En-Fr, the datasets are from WMT14 and we use newstest13 as the valid dataset and use newstest14 filtered by Stanford (Bojar et al., 2014) as the test dataset. For Zh-En, the datasets are the same as Li et al. (2020). We utilize the approach mentioned in Section 3.2 to construct synthetic bilingual parallel sentence pairs based on all the datasets above. To set up the data augmentation strategy, we combine original training datasets and their corresponding synthetic datasets. As Table 1 shows, we get an 8M English-German dataset based on the WMT14 En-De, a 72M English-French dataset based on WMT14 En-Fr, and a 4M Chinese-English dataset based on a Zh-En dataset mentioned above. Valid sets are obtained the same as training datasets. For all datasets mentioned above, we use Moses toolkit to tokenize and clean data. Besides, we use BPE (Sennrich et al., 2015) to process all the source and target sentences. We train and evaluate the following systems for comparison. Transformer. We set Transformer (Vaswani et al., 2017) using fairseq (Ng et al., 2019). The Transformer model is trained on the bilingual datasets: WMT14 En-De, WMT14 En-Fr, and the Zh-En dataset. LCDIMT. Since LCD-based decoding is widely used in IMT scenario (Weng et al., 2019; Huang et al., 2021), we build an LCD-based IMT as our strong baseline. Because the original LCD algorithm is very slow, we implement its efficient version (Post and Vilar, 2018) which achieves comparable translation quality to the original version but 5x speedup in decoding. Our System. Our BiTiIMT model is based on the base Transformer architecture and trained on the synthetic datasets mentioned in Section 3.3. All baselines (Transformer and LCDIMT) and the BiTIIMT models are based on the architecture with d model = 512 , d hidden = 2048 , n heads = 8 , n layers = 6 , and p dropout = 0 . 1 . We use the Adam Optimizer (Kingma and Ba, 2015) with  1 = 0 . 9 and  2 = 0 . 98 to train our models. We adopt a warm-up of 10,000 steps and set the initial learning rate to 0.0007. We set the maximum tokens in batch to 4096, and we share both source and target embeddings for all models. Training stops until the maximum update is 400,000 and the checkpoint used for testing is selected according to its performance on the valid dataset. We train all models on 16 NVIDIA V100 Tensor Core GPUs. We use a beam size of 10 throughout our experiments. Following prior work (Cheng et al., 2016; Weng et al., 2019; Zhao et al., 2020; Huang et al., 2021), two criteria are used to evaluate INMT systems: one is translation quality and the other is efficiency to yield the translation. We employ BLEU (Pa-pineni et al., 2002) to measure translation quality, and human editing cost to measure efficiency, which is calculated as the edit distance by counting deletions on word level and insertions on char level. In addition, we take the decoding time into account because it is directly related to the latency for human translators, which is critical for user experience. We conducted two different simulated experiments, including IMT with a single iteration and IMT with multiple iterations, to validate the effectiveness of our method in terms of translation quality and editing and decoding costs mentioned above. Since IMT with a single iteration can be seen as machine translation with lexical constraints where human interactions are considered as constraints, we first conduct an experiment to evaluate the performance of BiTIIMT by following Hokamp and Liu (2017); Post and Vilar (2018). We use the reference as oracle to sample constraints for all datasets and we consider five settings: they include gradually increasing constraint segments from one to five. In more detail, we randomly add a constraint segment with a random length between 1 to 3 for each setting. To ensure fairness, the constraints provided to both BiTIIMT and LCDIMT are exactly the same. We measure decoding cost by using the time required to translate the whole test set with a batch size of one (excluding the time of model loading). As shown in Table 2, with only one constraint segment, both BiTIIMT and LCDIMT obtain substantial improvements compared with Transformer and BiTIIMT significantly outperforms LCDIMT with a margin of +2.2, +4.63, and +1.12 BLEU points for En-De, En-Fr, and Zh-En tasks, respectively. This finding clearly verifies our hypothesis: BiTIIMT indeed makes full use of the constraint segments and thus yields better translations than LCDIMT. Table 2 also reports the results of relative decoding cost with respect to the Transformer baseline. As we can see, in the first setting with one constraint segment, BiTIIMT achieves modest speedup in decoding time compared with the Transformer baseline. With the growing number of constraint segments, BiTIIMT keeps reducing the decoding time, and it gives 0.27x decrease in decoding time cost on En-De when running on five constraint segments. Meanwhile, the time cost of LCDIMT keeps growing, and it is almost twice as that of the baseline in the 5th setting. Similar results can be found on En-Fr and Zh-En datasets. It is worth noting that the decoding efficiency of LCDIMT seems not an issue for one iteration. In fact, decoding efficiency indeed is a severe issue for multiple iterations as in real-world scenarios, where more constraint segments are involved especially at the late stage of iterations. We now turn to the evaluation of BiTIIMT in a simulated IMT scenario where multiple iterations of interactions are allowed. To simulate the human interactive process of IMT, at each iteration, we use the reference as oracle and match the oracle with a translation from each system to calculate  Y . Specifically, we delete unmatched words in the translation as simulated deletion and add a word from oracle as a simulated revision. By using the edit distance algorithm, which includes only deletion and revision operators, we can figure out  Y given a translation and its reference. We use the edit cost mentioned in Section 4.1.3 to qualify human interaction cost. Figure 5 shows BLEU scores and interaction costs along with all iterations on En-De, En-Fr, and Zh-En datasets. As expected, BLEU scores consistently increase with the increase of interaction costs. As we can see on En-De, BiTIIMT obtains improvements of about 5 BLEU points over the baseline LCDIMT when using similar human interaction costs. The BLEU gap between BiTIIMT and LCDIMT is further enlarged in the late stage of the interactive process. Results on En-Fr and Zh-En give similar conclusions. These facts show that BiTIIMT outperforms LCDIMT as it can reduce the human interaction cost to get a satisfying translation. Figure 6 reports the accumulated decoding time cost on three datasets. As we can see, results on three datasets give similar conclusions with the conclusions in section 4.2.2 that BiTIIMT has a lower decoding time cost compared to LCDIMT for all interactive iterations. Furthermore, the accumulated decoding time cost of BiTIIMT after 5 iterations is lower than the time cost of LCDIMT with only 2 iterations. The facts indicate that BiTIIMT has its outstanding advantage in efficiency. We conduct two kinds of IMT experiments to validate the effectiveness of BiTIIMT in a real-world scenario. First, we use the post-editing data from a valid dataset of WMT21 Automatic Post-Editing Shared Task on En-De (Sharma et al., 2021), where the words are edited by humans are natural constraints instead of simulated constraints. By using these constraints, we compare the proposed IMT method with baselines involve a single iteration of human-machine interactions. Results in table 3 show that BiTIIMT can obtain much better translations than LCDIMT. Since in this dataset there are many constraints edited by humans and the post-edited translation is used as the reference, the improvements of BiTIIMT over LCDIMT are substantial in terms of BLEU (up to 9 BLEU points). Furthermore, we conduct another real-world experiment that involves multiple iterations of human-machine interactions. Specifically, we randomly sample 200 sentences from the Zh-En test set and then ask two professional human translators to interact with both systems. Translators are asked to do interactions (deletions, revisions, and insertions) multiple times until they get a satisfactory translation. We compared LCD-based and BiTI-based IMT systems on averaged BLEU, averaged decoding time cost, and averaged editing cost of deletions, revisions, and insertions supplied by human translators. As shown in Table 4, BiTIIMT can reach higher BLEU points with less decoding time cost, editing cost, as well as fewer interaction rounds. Note that our BLEU gains over LCDIMT in Table 4 are relatively small compared with those in Table 3. One main reason is that the constraints edited by human translators may not appear in the reference translation. Effect of Data Augmentation As the description in Section 3.2.2, we train our model (#Aug-mented in Table 5) on the augmented data which includes synthetic bilingual data D = {(cid:104) X,  Y , Y b (cid:105)} and their corresponding bilingual parallel data D (cid:48) = {(cid:104) X,  , Y (cid:105)} by data augmentation. For com-1964 Method BLEU Transformer 41.92 LCDIMT 46.96 BiTIIMT 56.02 Table 3: Results on the real-world dataset from WMT21 En-De Automatic Post-Editing shared task. parison, we train two additional models: one is on D only (#Synthetic Only in Table 5) and the other is trained on D (cid:48) (#Raw in Table 5). We compare all these three models according to their BLEU on the Zh-En dataset. Note that in this experiment, we do not provide any human interactions to all three models, i.e.,  Y =  , during testing. Table 5 summarizes the results of the three models. The model trained on the only synthetic dataset almost collapses on automatic machine translation task: it is worse than # Raw model by a substantial margin of 18 BLEU points. In addition, it can be found that the model trained on the augmented dataset achieves +0.68 BLEU improvements over # Raw model. Although such improvements are not that large, it still shows that data augmentation or the application of multiple task learning plays a critical role in making BiTiIMT successful. Effect of Random Sampling Strategy When training the model for BiTIMT in section 3.2.2, we employ a random strategy to sample  Y such that it contains a random number of constraint segments. We are also curious about the effect of such a random strategy. In comparison, we fix the number of constraint segments and then train three models for k = { 1 , 3 , 5 } . Under all five settings as in Section 4.2.1, we compare BiTIIMT with these three models on the Zh-En task. Figure 7 shows that, by training on data with a diverse number of constraints, our model achieves increasing BLEU on all five settings while the BLEU of the model ( k = 1 ) gradually decreases and another two models could not give continuous improvement. Results suggest that our random sampling strategy assists BiTIIMT to translate with a various number of constraints. Since the period of statistical machine translation (SMT), IMT has been widely exploited to reduce human effort by using human's feedback to improve translation quality (Foster et al., 1997; Langlais et al., 2000; Simard et al., 2007; Barrachina et al., 2009; Gonzlez-Rubio et al., 2013; Cheng et al., 2016; Li et al., 2021). Recently, with the development of NMT (Bahdanau et al., 2015; Vaswani et al., 2017), researchers turned to employ IMT on it (Hokamp and Liu, 2017; Wang et al., 2020a). A classical type of IMT uses a left-to-right sentence completing framework proposed in Langlais et al. (2000), in which human translators can only revise the translation generated by models from left to right. Generally, the text portion from the beginning to the current modified part is called prefix, and the system will generate a new translation based on the given prefix (Sanchis-Trilles et al., 2014; Peris et al., 2017a; Knowles and Koehn, 2016). Cheng et al. (2016) propose a pick-revise framework that enables translators to do revisions on arbitrary positions to improve efficiency. Huang et al. (2021) allow users to make any interaction at random positions by using LCD algorithms (Hokamp and Liu, 2017; Post and Vilar, 2018) in the decoding stage which can integrate lexical constraints into translation. However, LCD can not achieve a win-win result in terms of decoding speed and translation quality. Weng et al. (2019) propose a bidirectional IMT framework on top of LCD, which could fix mistakes by using two constrained decoding procedures with opposite directions. However, it needs to train two decoders, and in each con-1965 Methods BLEU Decoding Time Cost Editing Cost Rounds Transformer 45.46 0.92 -LCDIMT 51.36 1.44 19.79 1.46 BiTIIMT 53.48 0.27 15.815 1.27 Table 4: Results in the real-world IMT scenario for different methods. strained decoding, the model can only use part of the constraints supplied by translators, making it inefficient both in using human knowledge and decoding speed. Instead, BiTIIMT puts all constraints into a template as part of the input, which makes it possible for the model to use all human knowledge and meanwhile fix mistakes automatically in the whole sentence. Other works (Alkhouli et al., 2019; Song et al., 2020; Chen et al., 2021) apply alignment information to improve the decoding efficiency of LCD. Alkhouli et al. (2019) use alignment extracted by vanilla transformer, which is poor as argued by Garg et al. (2019). Song et al. (2020) apply an external aligner to train the alignment module. These works can only perform constrained decoding based on a constraint pair, which means a burden for human translators. In order to address the issue of decoding speed for LCD, some works use a non-autoregressive approach to integrate constraints. Susanto et al. (2020) propose Levenshtein Transformer (Gu et al., 2019) to inject terminology constraints at inference time. Xu and Carpuat (2021) propose a novel re-position operator to replace deletion in Levenshtein Transformer to exploit lexical constraints more effectively and efficiently. However, non-autoregressive models are still worse than autoregressive models in translation quality currently. Compared to these efforts in NAT, BiTIIMT is essentially based on an auto-regressive translation model. Traditional IMT systems often use LCD to incorporate manually revised words into translations. In this paper, we propose BiTIIMT, a novel IMT method that outperforms LCD-based IMT in both translation quality and efficiency. The key to BiTIIMT is the bilingual text-infilling task which extends text-infilling from a monolingual setting to a bilingual one. We cast this task as a sequence-to-sequence task and propose a simple yet effective solution to address it. Experiments show that BiTIIMT achieves a significantly improved efficiency in the area of IMT. We would like to thank the anonymous reviewers for their insightful comments. Shujian Huang and Lemao Liu are the corresponding co-authors. This work is supported by National Science Foundation of China (No. U1836221, 6217020152).
abstain abstain abstain abstain method abstain method method abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain method objective abstain abstain method objective abstain objective objective other other other other other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method abstain method abstain abstain abstain abstain method abstain abstain other abstain abstain abstain abstain method method abstain abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result other other other other other other other other other other other other other other other other other other other other abstain objective abstain objective abstain other other other