Simultaneous translation has many important application scenarios and attracts much attention from both academia and industry recently. Most existing frameworks, however, have dif-ficulties in balancing between the translation quality and latency, i.e., the decoding policy is usually either too aggressive or too conser-vative. We propose an opportunistic decoding technique with timely correction ability, which always (over-)generates a certain mount of extra words at each step to keep the audience on track with the latest information. At the same time, it also corrects, in a timely fashion, the mistakes in the former overgenerated words when observing more source context to ensure high translation quality. Experiments show our technique achieves substantial reduction in latency and up to +3.1 increase in BLEU, with revision rate under 8% in Chinese-to-English and English-to-Chinese translation. Simultaneous translation, which starts translation before the speaker finishes, is extremely useful in many scenarios, such as international conferences, travels, and so on. In order to achieve low latency, it is often inevitable to generate target words with insufficient source information, which makes this task extremely challenging. Recently, there are many efforts towards balancing the translation latency and quality with mainly two types of approaches. On one hand, Ma et al. (2019a) propose very simple frameworks that decode following a fixed-latency policy such as wait-k . On the other hand, there are many attempts to learn an adaptive policy which enables the model to decide READ or WRITE action on the fly using various techniques such as reinforcement learning (Gu et al., 2017; Alinejad et al., 2018; Grissom II  These authors contributed equally. et al., 2014), supervised learning over pseudo-oracles (Zheng et al., 2019a), imitation learning (Zheng et al., 2019b), model ensemble (Zheng et al., 2020) or monotonic attention (Ma et al., 2019d; Arivazhagan et al., 2019). Though the existing efforts improve the performance in both translation latency and quality with more powerful frameworks, it is still difficult to choose an appropriate policy to explore the optimal balance between latency and quality in practice, especially when the policy is trained and applied in different domains. Furthermore, all existing approaches are incapable of correcting the mistakes from previous steps. When the former steps commit errors, they will be propagated to the later steps, inducing more mistakes to the future. Inspired by our previous work on speculative beam search (Zheng et al., 2019c), we propose an opportunistic decoding technique with timely correction mechanism to address the above problems. As shown in Fig. 1, our proposed method always decodes more words than the original policy at each step to catch up with the speaker and reduce the latency. At the same time, it also employs a timely correction mechanism to review the extra outputs from previous steps with more source context, and revises these outputs with current preference when there is a disagreement. Our algorithm can be used in both speech-to-text and speech-to-speech simultaneous translation (Oda et al., 2014; Bangalore et al., 2012; Yarmoham-madi et al., 2013). In the former case, the audience will not be overwhelmed by the modifications since we only review and modify the last few output words with a relatively low revision rate. In the later case, the revisable extra words can be used in look-ahead window in incremental TTS (Ma et al., 2019b). By contrast, the alternative retranslation strategy (Arivazhagan et al., 2020) will cause non-local revisions which makes it impossible to be used in incremental TTS. We also define, for the first time, two metrics for revision-enabled simultaneous translation: a more general latency metric Revision-aware Average Lagging (RAL) as well as the revision rate . We demonstrate the effectiveness of our proposed technique using fixed (Ma et al., 2019a) and adaptive (Zheng et al., 2019a) policies in both Chinese-to-English and English-to-Chinese translation. Full-sentence NMT. The conventional full-sentence NMT processes the source sentence x = ( x 1 , ..., x n ) with an encoder, where x i represents an input token. The decoder on the target side (greedily) selects the highest-scoring word y t given source representation h and previously generated target tokens, y <t = ( y 1 , ..., y t  1 ) , and the final hypothesis y = ( y 1 , ..., y t ) with y t = <eos> has the highest probability: p ( y | x ) = (cid:81) | y | t =1 p ( y t | x , y <t ) (1) Simultaneous Translation. Without loss of generality, regardless the actual design of policy, simultaneous translation is represented as: p g ( y | x ) = (cid:81) | y | t =1 p ( y t | x (cid:54) g ( t ) , y <t ) (2) where g ( t ) can be used to represent any arbitrary fixed or adaptive policy. For simplicity, we assume the policy is given and does not distinguish the difference between two types of policies. Opportunistic Decoding. For simplicity, we first apply this method to fixed policies. We define the original decoded word sequence at time step t with y t , which represents the word that is decoded in time step t with original model. We denote the additional decoded words at time step t as  y (cid:54) w t = ( y 1 t , ..., y wt ) , where w denote the number of extra decoded words. In our setting, the decoding process is as follows: p g ( y t   y (cid:54) w t | x (cid:54) g ( t ) ) = p g ( y t | x (cid:54) g ( t ) ) (cid:81) wi =1 p g ( y it | x (cid:54) g ( t ) , y t   y <it ) (3) where  is the string concatenation operator. We treat the procedure for generating the extra decoded sequence as opportunistic decoding, which prefers to generate more tokens based on current context. When we have enough information, this opportunistic decoding eliminates unnecessary latency and keep the audience on track. With a certain chance, when the opportunistic decoding tends to aggressive and generates inappropriate tokens, we need to fix the inaccurate token immediately. Timely Correction. In order to deliver the correct information to the audience promptly and fix previous mistakes as soon as possible, we also need to review and modify the previous outputs. At step t + 1 , when encoder obtains more information from x (cid:54) g ( t ) to x (cid:54) g ( t +1) , the decoder is capable to generate more appropriate candidates and may revise and replace the previous outputs from opportunistic decoding. More precisely,  y (cid:54) w t and y t +1   y (cid:54) w  1 t +1 are two different hypothesis over the same time chunk. When there is a disagreement, our model always uses the hypothesis from later step to replace the previous commits. Note our model does not change any word in y t from previous step and it only revise the words in  y (cid:54) w t . Modification for Adaptive Policy. For adaptive policies, the only difference is, instead of committing a single word, the model is capable of generating multiple irreversible words. Thus our proposed methods can be easily applied to adaptive policies. Correction with Beam Search. When the model is committing more than one word at a time, we can use beam search to further improve the translation quality and reduce revision rate (Mur-ray and Chiang, 2018; Ma et al., 2019c). The decoder maintains a beam B kt of size b at step t , which is ordered list of pairs bsh (cid:2802)(cid:1355) Bush z  ngt  ng (cid:3035)(cid:5497) President de (cid:4913) of Ji  ng (cid:4079) Jiang f  yn (cid:1870)(cid:6522) speech bi  osh (cid:6418)(cid:5135) express Zmn (cid:4162)(cid:4038) Zemin di (cid:2642) to Jiang Zemin expressed his welcome to his agreement to President 1 2 3 4 5 6 7 8 9 zntng (cid:6761)(cid:1910) agreement  decoding time t = 4 t = 5 expressed  (cid:1157) bngqi  (cid:2842)(cid:1260) and 10 11 Jiang Zemin his to President t = 6 expressed Jiang Zemin Bush agreement Figure 2: The decoder generates target word y 4 = his and two extra words welcome to at step t = 4 when input x 9 = z`antong (agreement) is not available yet. (cid:104) hypothesis, probability (cid:105) , where k denotes the k th step in beam search. At each step, there is an initial beam B 0 t = [ (cid:104) y t  1 , 1 (cid:105) ] . We denote one-step transition from the previous beam to the next as B k +1 t = next b 1 ( B kt ) = b top {(cid:104) y (cid:48)  v, u  p ( v | x (cid:54) g ( t ) , y (cid:48) ) (cid:105) | (cid:104) y (cid:48) , u (cid:105) B k t } where top b (  ) returns the top-scoring b pairs. Note we do not distinguish the revisable and non-revisable output in y (cid:48) for simplicity. We also define the multi-step advance beam search function with recursive fashion as follows: next bi ( B kt )=next b 1 (next bi  1 ( B kt )) When the opportunistic decoding window is w at decoding step t , we define the beam search over w + 1 (include the original output) as follows: (cid:104) y (cid:48) t , u t (cid:105) = top 1 (cid:0) next bn + w ( B 0 t ) (cid:1) (4) where next bn + w (  ) performs a beam search with n + w steps, and generate y (cid:48) t as the outputs which include both original and opportunistic decoded words. n represents the length of y t 4 Revision-aware AL and Revision Rate We define, for the first time, two metrics for revision-enabled simultaneous translation. ALAL is introduced in (Ma et al., 2019a) to measure the average delay for simultaneous translation. Besides the limitations that are mentioned in (Cherry and Foster, 2019), AL is also not sensitive to the modifications to the committed words. Furthermore, in the case of re-translation, AL is incapable to measure the meaningful latency anymore. We hereby propose a new latency, Revision-aware AL (RAL), which can be applied to any kind of translation scenarios, i.e., full-sentence translation, use re-translation as simultaneous translation, fixed and adaptive policy simultaneous translation. Note that for latency and revision rate calculation, we count the target side difference respect to the growth of source side. As it is shown in Fig. 3, there might be multiple changes for each output words during the translation, and we only start to calculate the latency for this word once it agrees with the final results. Therefore, it is necessary to locate the last change for each word. For a given source side time s , we denote the t th outputs on target side as f ( x (cid:54) s ) t . Then we are able to find the Last Revision (LR) for the t th word on target side as follows: LR ( t ) = argmax s< | x | (cid:0) f ( x (cid:54) ( s  1) ) t (cid:54) = f ( x (cid:54) s ) t (cid:1) From the audience point of view, once the former words are changed, the audience also needs to take the efforts to read the following as well. Then we also penalize the later words even there are no changes, which is shown with blue arrow in Fig. 3. We then re-formulate the LR ( t ) as follows (assume LR (0) = 0 ): 5 10 15 Revision-aware Average Lagging (zh en) 25.0 27.5 30.0 32.5 35.0 37.5 40.0 42.5 45.0 4 r e f BLEU w =5, b >1 w =3, b >1 w =1, b >1 w =5, b =1 w =3, b =1 w =1, b =1 w =0, b =1 29.6 0 5 10 15 Revision-aware Average Lagging (en zh) 10 12 14 16 18 20 22 24 1 r e f BLEU w =5, b >1 w =3, b >1 w =1, b >1 w =5, b =1 w =3, b =1 w =1, b =1 w =0, b =1 38.3 Figure 4: BLEU against RAL using waitk polocies. LR ( t ) = max { LR ( t  1) , LR ( t ) } (5) The above definition can be visualized as the thick black line in Fig. 3. Similar with original AL, our proposed RAL is defined as follows: RAL( x , y ) = 1  ( | x | )  ( | x | ) (cid:88) t =1 LR ( t )  t  1 r (6) where  ( | x | ) denotes the cut-off step, and r = | y | / | x | is the target-to-source length ratio. Since each modification on the target side would cost extra effort for the audience to read, we penalize all the revisions during the translation. We define the revision rate as follows: (cid:16) | x | 1 (cid:88) s =1 dist (cid:16) f ( x (cid:54) s ) , f ( x (cid:54) s +1 ) (cid:17)(cid:17)(cid:46)(cid:16) | x | (cid:88) s =1 | f ( x (cid:54) s ) | (cid:17) where dist can be arbitrary distance measurement between two sequences. For simplicity, we design a modified Hamming Distance to measure the difference: dist ( a, b ) = hamming (cid:0) a, b | a | (cid:104) pad (cid:105) max( | a || b | , 0) (cid:1) where (cid:104) pad (cid:105) is a padding symbol in case b is shorter than a . Datasets and Implementation We evaluate our work on Chinese-to-English and English-to-Chinese simultaneous translation tasks. We use the NIST corpus (2M sentence pairs) as the training data. We first apply BPE (Sennrich et al., 2015) on all texts to reduce the vocabulary sizes. For evaluation, we use NIST 2006 and NIST 2008 as our dev and test sets with 4 English references. We re-implement waitk model (Ma et al., 2019a) and adaptive policy (Zheng et al., 2019a). We use Transformer (Vaswani et al., 2017) based wait-k model and pre-trained full-sentence model for learning adaptive policy. Performance on Waitk Policy We perform experiments using opportunistic decoding on wait-k policies with k  { 1 , 3 , 5 , 7 , 9 } , opportunistic window w  { 1 , 3 , 5 } and beam size b  { 1 , 3 , 5 , 7 , 10 , 15 } . We select the best beam size for each policy and window pair on dev-set. We compare our proposed method with a baseline called re-translation which uses a full-sentence NMT model to re-decode the whole target sentence once a new source word is observed. The final output sentences of this method are identical to the full sentence translation output with the same model but the latency is reduced. Fig. 4 (left) shows the Chinese-to-English results of our proposed algorithm. Since our greedy opportunistic decoding doesn't change the final output, there is no difference in BLEU compared with normal decoding, but the latency is reduced. However, by applying beam search, we can achieve 3.1 BLEU improvement and 2.4 latency reduction on wait-7 policy. Fig. 4 (right) shows the English-to-Chinese results. Compare to the Chinese-to-English translation results in previous section, there is comparatively less latency reduction by using beam search because the output translations are slightly longer which hurts the latency. As shown in Fig. 5(right), the revision rate is still controlled under 8%. Fig. 5 shows the revision rate with different window size on waitk policies. In general, with opportunity window w  5 , the revision rate of our proposed approach is under 8% , which is much lower than re-translation. Performance on Adaptive Policy Fig. 6 shows the performance of the proposed algorithm on adaptive policies. We use threshold   { 0 . 55 , 0 . 53 , 0 . 5 , 0 . 47 , 0 . 45 } . We vary beam size b  { 1 , 3 , 5 , 7 , 10 } and select the best one on dev-set. Comparing with conventional beam search on consecutive writes, our decoding algorithm achieves even much higher BLEU and less latency. 5.1 Revision Rate vs. Window Size 1 3 5 7 10 15 Beam Size (zh en) 0.5 1.0 1.5 2.0 2.5 3.0 3.5 R e v i s i o n R a t e wait-1 wait-3 wait-5 wait-7 wait-9 Figure 7: Revision rate against beam size with window size of 3 and different waitk policies. We further investigate the revision rate with different beam sizes on waitk policies. Fig. 7 shows that the revision rate is higher with lower waitk policies. This makes sense because the low k policies are always more aggressive and easy to make mistakes. Moreover, we can find that the revision rate is not very sensitive to beam size. We have proposed an opportunistic decoding timely correction technique which improves the latency and quality for simultaneous translation. We also defined two metrics for revision-enabled simultaneous translation for the first time. L. H. was supported in part by NSF IIS-1817231.
abstain abstain objective abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain method method abstain abstain objective abstain abstain abstain other abstain method abstain abstain method abstain method method abstain method abstain method abstain abstain method abstain abstain abstain abstain abstain abstain other abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain other abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result objective objective other