Document-level contextual information has shown benefits to text-based machine translation, but whether and how context helps end-to-end (E2E) speech translation (ST) is still under-studied. We fill this gap through extensive experiments using a simple concatenation-based context-aware ST model, paired with adaptive feature selection on speech encodings for computational efficiency. We investigate several decoding approaches, and introduce in-model ensemble decoding which jointly performs documentand sentence-level translation using the same model. Our results on the MuST-C benchmark with Transformer demonstrate the effectiveness of context to E2E ST. Compared to sentence-level ST, context-aware ST obtains better translation quality (+0.18-2.61 BLEU), improves pronoun and homophone translation, shows better robustness to (artificial) audio segmentation errors, and reduces latency and flicker to deliver higher quality for simultaneous translation. 1 1 Introduction Document-level context often offers extra informative clues that could improve the understanding of individual sentences. Such clues have been proven effective for textual machine translation (MT), particularly in handling translation errors specific to discourse phenomena, such as inaccurate corefer-ence of pronouns (Guillou, 2016) and mistranslation of ambiguous words (Rios et al., 2017). Besides, ensuring consistency in translation is virtually impossible without document-level context as well (Voita et al., 2019). Analogous to MT, speech translation (ST) also suffers from these translation issues, and super-sentential context could in fact be more valuable to ST because 1) homophones 1 Source code is available at https://github.com/ bzhangGo/zero . Figure 1 : Overview of the concatenation-based context-aware ST. y n denotes the n -th target sentence in a document; x n denotes the speech encodings extracted from the n -th audio segment. We use dashed gray box to indicate the concatenation operation. < s > : sentence separator symbol. and acoustic noise bring additional ambiguity to ST, and 2) a common use case in ST is simultaneous translation, where the system has to output translations of sentence fragments, and may have to predict future input to account for word order differences between the source and target language (Grissom II et al., 2014). Both for ambiguity from the acoustic signal, and operating on small sentence fragments, we hypothesize that access to extra context 2 will be beneficial. Although recent studies on ST have achieved promising results with end-to-end (E2E) models (Anastasopoulos and Chiang, 2018; Di Gangi et al., 2019; Zhang et al., 2020a; Wang et al., 2020; Dong et al., 2020), nevertheless, they mainly focus on sentence-level translation. One practical challenge when scaling up sentence-level E2E ST to the document-level is the encoding of very long audio segments, which can easily hit the computational bottleneck, especially with Transformers (Vaswani et al., 2017). So far, the research question of whether and how contextual information benefits E2E ST has received little attention. In this paper, we answer this question through extensive experiments by exploring a concatenation-2 By default, we use context to denote both sourceand target-side information from previous sentences. based context-aware ST model. Figure 1 illustrates our model, where neighboring source (tar-get) sequences are chained together into one sequence for joint translation. This paradigm only requires data-level manipulation, thus allowing us to reuse any existing sentence-level E2E ST models. Despite its simplicity, this approach successfully leverages contextual information to improve textual MT (Tiedemann and Scherrer, 2017; Bawden et al., 2018; Lopes et al., 2020), and here we adapt it to ST. As for the computational bottleneck, we shorten the speech encoding sequence via adaptive feature selection (Zhang et al., 2020b,a, AFS), which only retains a small subset of encodings (  16%) for each audio segment. We investigate several decoding methods, including chunk-based decoding and sliding-window based decoding. We also study an extension of the latter with the constraint of target prefix, where the prefix denotes the translation of previous context speeches. We find that using these methods sometimes results in misaligned translations , particularly when using the constraint. This issue manifests itself in mismatching sentence boundaries and producing overand/or under-translation, which greatly hurts sentence-based evaluation metrics. To avoid such misalignments, we introduce in-model ensemble decoding (IMED) to regularize the document-level translation with its sentence-level counterpart. Note that we use the same context-aware ST model here for both types of translation  that's why we call it in-model ensemble. We adopt Transformer (Vaswani et al., 2017) for experiments with the MuST-C dataset (Di Gangi et al., 2019). We study the impact of context on translation in different settings. Our results demonstrate the effectiveness of contextual modeling. Our main findings are summarized below:  Incorporating context improves overall translation quality (+0.18-2.61 BLEU) and benefits pronoun translation across different language pairs, resonating with previous findings in textual MT (Miculicich et al., 2018; Huo et al., 2020). In addition, context also improves the translation of homophones. ST models with contexts suffer less from (ar-tificial) audio segmentation errors. Contextual modeling improves translation quality and reduces latency and flicker for simultaneous translation under re-translation strategy (Arivazhagan et al., 2020a). Our work is inspired by pioneer studies on context-aware textual MT. Context beyond the current sentence carries information whose importance for translation cohesion and coherence has long been posited (Hardmeier et al., 2012; Xiong and Zhang, 2013). With the rapid development of neural MT and also available document-level textual datasets, research in this direction gained great popularity. Recent efforts often focus on either advanced contextual neural architecture development (Tiede-mann and Scherrer, 2017; Kuang et al., 2018; Miculicich et al., 2018; Zhang et al., 2018, 2020c; Kang et al., 2020; Chen et al., 2020; Ma et al., 2020a; Zheng et al., 2020) and/or improved analysis and evaluation targeted at specific discourse phenomena (Bawden et al., 2018; Laubli et al., 2018; Guillou et al., 2018; Voita et al., 2019; Kim et al., 2019; Cai and Xiong, 2020). We follow this research line, and adapt the concatenation-based contextual model (Tiedemann and Scherrer, 2017; Bawden et al., 2018; Lopes et al., 2020) to ST. Our main interest lies in exploring the impact of context on ST. Developing dedicated contextual models for ST is beyond the scope of this study, which we leave to future work. Context-aware ST extends the sentence-level ST towards streaming ST which allows models to access unlimited previous audio inputs. Instead of improving contextual modeling, many studies on streaming ST aim at developing better sentence/-word segmentation policies to avoid segmentation errors that greatly hurt translation (Matusov et al., 2007; Rangarajan Sridhar et al., 2013; Iranzo-Sanchez et al., 2020; Zhang and Zhang, 2020; Arivazhagan et al., 2020b). Very recently, Ma et al. (2020b) proposed a memory augmented Transformer encoder for streaming ST, where the previous audio features are summarized into a growing continuous memory to improve the model's context awareness. Despite its success, this method ignores the target-side context, which turns out to have sig-nificant positive impact on ST in our experiments. Our study still relies on oracle sentence segmentation of the audio. The most related work to ours is (Gaido et al., 2020), which also investigated con-textualized translation and showed that context-aware ST is less sensitive to audio segmentation errors. While they exclusively focus on the robustness to segmentation errors, our study investigates the benefits of context-aware E2E ST more broadly. We extend the sentence-level ST with document-level context, by modeling up to C previous source/target segments/sentences for translation. Formally, given a pre-segmented audio (source document) A = (cid:0) a 1 , . . . , a N (cid:1) as well as its paired target document Y = (cid:0) y 1 , . . . , y N (cid:1) , the model is trained to maximize the following likelihood: log p ( Y | A ) = N (cid:88) n =1 log p (cid:0) y n | x n , C n y , C n x (cid:1) , (1) where x n = AFS ( a n ) , i.e. the speech encodings extracted via AFS (Zhang et al., 2020a). a n and y n denote the n -th audio segment and target sentence, respectively. N is the number of seg-ments/sentences in the document. C n x and C n y stand for the source and target context, respectively, i.e. { x n  i } Ci =1 and { y n  i } Ci =1 . Adaptive Feature Selection Audio segment is often converted into frame-based features for neural modeling. Different from text, each segment might contain hundreds or even thousands of such features, making contextual modeling computationally difficult. Zhang et al. (2020a) found that most speech encodings emitted by a Transformer-based audio encoder carry little information for translation, and their deletion even improves translation quality. We follow Zhang et al. (2020a) and perform AFS to only extract those informative encodings (  16%) optimized via sentence-level speech recognition with L 0 DROP (Zhang et al., 2020b). This greatly shortens the speech encoding sequence, thus enabling broader context exploration. the previous context ( C n x / C n y ) (Tiedemann and Scherrer, 2017; Bawden et al., 2018) as shown in Figure 1. After obtaining the AFS-based encodings ( x n ) for each audio segment, we concatenate those encodings of neighboring segments to form the source input. The same is applied to the target-side sentences, except for a separator symbol  < s >  inserted in-between sentences to distinguish sentence boundaries. 3 Such modeling enables us to use arbitrary encoder-decoder models for context-aware ST, such as the Transformer (Vaswani et al., 2017) used in this paper. Despite no dedicated hierarchical modeling (Miculicich et al., 2018), this paradigm still allows for intraand inter-sentence attention during encoding and decoding, which explicitly utilizes context for translation and has been proven successful (Lopes et al., 2020). Concatenation-based contextual modeling allows for different inference strategies with possible trade-offs between simplicity/efficiency and accuracy. We investigate the following inference strategies (see Figure 2): Chunk-based Decoding (CBD) CBD splits all audio segments in one document into nonoverlapping chunks, with each chunk concatenating C + 1 segments, as shown in Figure 2a. CBD directly translates each chunk, and then recovers sentence-level translation via the separator symbol  < s > . CBD is the most efficient inference strategy, only encoding/decoding each sentence once, but it might suffer from misaligned translation , 3 Note that we did not add similar boundary information to audio segments, because AFS implicitly captures these signals through independent segment encoding. producing more or fewer sentences than the input segments. We simply drop the extra generated sentences and replace the missing ones with  < unk >  when computing sentence-based evaluation metrics. Also, CBD introduces an independence assumption between chunks. Sliding Window-based Decoding (SWBD) SWBD avoids such inter-chunk independence by sequentially translating each audio segment ( x n ), together with its corresponding previous source context ( C n x ). We distinguish two variants of SWBD. The first variant, SWBD, translates the concatenated segments and regards the last generated sentence as the translation of the current segment while discarding all other generations (Figure 2b). Note that this might introduce inconsistencies between the output produced at a time step, and the one used as target context in future time steps. By contrast, the second variant, SWBD-Cons, leverages the previously generated (up to C ) sentences as a decoding constraint, based on which the model only needs to generate one sentence (Figure 2c). In-Model Ensemble Decoding (IMED) We observe that SWBD still suffers from misaligned translation , where the translation of the current segment might contain information from previous segments. We introduce IMED to alleviate this issue as shown in Figure 2d. IMED extends SWBD-Cons by interpolating the document-level prediction ( p d ) with the sentence-level prediction ( p s ) as follows: p s ( y nt | y n<t , x n ) + (1   ) p d ( y nt |C ) , (2) where C = {C n x , C n y , x n , y n<t } ,  is a hyperparameter, y nt denotes the t -th target word in sentence y n , and both predictions are based on the same model  . Intuitively, the sentence-level translation acts as a regularizer, avoiding the overor under-translation. Note IMED with  = 0 corresponds to SWBD-Cons. We use the MuST-C dataset (Di Gangi et al., 2019) for experiments, which was collected from English TED talks and covers translations from English to 8 different languages, including German (De), Spanish (Es), French (Fr), Italian (It), Dutch (Nl), Portuguese (Pt), Romanian (Ro) and Russian (Ru). MuST-C offers a standard training, development and test set split for each language pair, with each dataset consisting of English audio, English transcriptions and their translations. Each training set contains transcribed speeches of  452 hours with  252K utterances on average. We report results on tst-COMMON, whose size ranges from 2502 (Es) to 2641 (De) utterances. We perform our major study on MuST-C En-De. To construct acoustic features, for each audio segment, we extract 40-channel log-Mel filterbanks using overlapping windows of 25 ms and step size of 10 ms. We enrich these features with their first and second-order derivatives, followed by mean subtraction and variance normalization. Following Zhang et al. (2020a), we perform nonoverlapping feature stacking to combine the features of three consecutive frames. All the texts are tokenized and truecased (Koehn et al., 2007), with out-of-vocabulary words handled by BPE segmentation (Sennrich et al., 2016), using 16K merging operations. Model Settings and Evaluation Our context-aware ST follows Transformer base (Vaswani et al., 2017): 6 layers, 8 attention heads, and hidden/feed-forward size 512/2048. We use Adam (  1 = 0 . 9 ,  2 = 0 . 98 ) (Kingma and Ba, 2015) for parameter updates with label smoothing of 0.1. We use the same learning rate schedule as Vaswani et al. (2017) and set the warmup step to 4K. We apply dropout to attention weights and residual connections with a rate of 0.2 and 0.5, respectively. By default, we set C = 2 and  = 0 . 5 . Following (Zhang et al., 2020a), we apply AFS( (cid:15) =  0 . 1 ,  = 2 / 3 ) to both temporal and feature dimensions for feature selection, which prunes out  84% speech encodings. We initialize our context-aware ST with the sentence-level Baseline, i.e. ST+AFS, and then finetune the model for 20K steps based on the concatenation method with a batch size of around 40K subwords. 4 We adopt beam search for decoding, with a beam size of 4 and length penalty of 0.6. We average the last 5 checkpoints for evaluation. We measure general translation quality with tokenized case-sensitive BLEU (Papineni et al., 2002) and also report the detokenized one via sacreBLEU (Post, 2018) 5 for cross-paper comparison. We calculate BLEU based on sentences unless oth-4 Our experiments show that such initialization eases the learning of long inputs and improves the convergence of context-aware ST. 5 signature: BLEU+c.mixed+#.1+s.exp+tok.13a+v.1.3.6 ID Model BLEU APT 1 Baseline (ST+AFS) 22.38 (27.40) 60.77 2 Ours + CBD 22.72 (27.95) 62.31 3 Ours + SWBD 22.70 (28.02) 62.83 4 Ours + SWBD-Cons 22.11 (27.98) 60.94 5 Ours + IMED 22.86 ( 28.03 ) 62.56 6 1 + 20K-step finetuning 22.02 (27.00) 61.58 7 5 +  = 1 . Table 1 : Case-sensitive tokenized BLEU and APT for different models and settings on MuST-C En-De test set. Numbers in bracket denote document -based BLEU. lp : the length penalty for beam search decoding. w/o C n y : models that are trained without target-side context. Best results are highlighted in bold. Note C = 2 ,  = 0 . 5 and lp = 0.6 by default. erwise specified. We use APT (Miculicich Werlen and Popescu-Belis, 2017), the accuracy of pronoun translation, as an approximate proxy for document-level evaluation. Word alignment required by APT is automatically extracted via fast align (Dyer et al., 2013) with the strategy  grow-diag-final-and . Does context improve translation? Yes, but the decoding method matters for context-aware ST. Table 1 summarizes the results. Our model with IMED outperforms Baseline by +0.48 BLEU (sig-nificant at p < 0.05) 6 and +1.79 APT (1  5), clearly showing the benefits from contextual modeling. Although SWBD-Cons yields worse sentence-based BLEU (-0.27, 1  4), it still beats Baseline in document-based BLEU (+0.58) and pronoun translation (+0.17 APT). The reason behind this inferior BLEU partially lies in misaligned translation (see Table 8 in Appendix for example). We observe that SWBD-Cons sometimes segments its output in a way that is misaligned to the reference segmentation. This also hurts CBD, where CBD produces mismatched sentences for around 1.8% cases. This is only a problem if we rely on the sentence-level alignment for BLEU, but not when we measure document-based BLEU (in brackets), where translations in one document are concatenated into a sequence for BLEU calculation. Overall, SWBD 6 We perform significance test using bootstrap-hypothesis-difference-significance.pl in moses (Koehn et al., 2007). and IMED are more stable and perform the best, and SWBD surpasses Baseline by 2.06 APT (1  3). We will proceed with using IMED and SWBD for more reliable results with APT and later analysis. Since we finetune our model based on the pre-trained Baseline, directly comparing with Baseline might be unfair. To offset its influence, we continue to train Baseline for the same 20K steps, following the settings in Section 5.1. Results show that this extra training (1  6) slightly deteriorates BLEU (-0.36) and only explains part of the improvement in APT (+0.81). Therefore, the gain brought by SWBD and IMED does not come from longer training. However, we do observe that initializing from the sentence-level Baseline benefits context-aware ST, compared to directly training context-aware ST from the AFS model (13  3, 14  4). Apart from faster convergence and higher quality, another benefit of this finetuning is that the trained context-aware ST still carries the ability to translate individual sentences. Table 1 shows that using context-aware ST for sentence-level translation (1  7) yields similar BLEU to Baseline (+0.04) but surprisingly much better pronoun translation (+1.19), although it still underperforms SWBD and IMED. The fact that we can perform sentence-level ST using the same context-aware ST model indicates that it can be useful for ensembling, as confirmed by the effectiveness of IMED. Upon closer inspection, we find that context-aware ST prefers to produce longer translations than Baseline. To control for the effects of output length on BLEU differences, we experiment with larger length penalty ( lp : 0.6  1.0) to beam search. Results in Table 1 show that biasing the decoding greatly improves sentence-level ST (1  8), achieving performance on par with context-aware ST (when lp is 0.6) in terms of BLEU with similar translation lengths but still falling short of pronoun translation (-0.94 APT, 8  3). In addition, we observe that context-aware ST also benefits from decoding with larger length penalty, beating all sentence-level ST models (3  9, 5  10). Particularly, SWBD with lp of 1.0 delivers the best BLEU of 22.97 and APT of 63.51 (3  9). Note we adopt lp of 0.6 for the following experiments. Does target-side context matter for context-aware ST? Yes, it matters a lot. By default, we utilize both sourceand target-side context for contextual modeling. Removing the target-side part (also at training), as shown in Table 1 (11, 12), sub-Model BLEU APT SWBD 22.70 62.83 SWBD + Random C n x 22.31 61.16 IMED 22.86 62.56 IMED + Random C n x 21.83 59.95 IMED + Random C n y 21.99 60.01 IMED + Random C n y & C n y 21.76 59.67 Table 2 : Case-sensitive tokenized BLEU and APT for context-aware ST with random source/target context on MuST-C En-De test set. We report average performance over three runs with different random seeds. C = 2 ,  = 0 . 5 . Incorrect context hurts our model. stantially weakens translation quality, even leading to worse performance than Baseline. Apart from offering direct target-side translation clues, we argue that the target-side context also enforces the context-aware ST to utilize the source-side context for translation, thus benefiting its training. This observation echoes with several previous studies on textual translation (Bawden et al., 2018; Huo et al., 2020; Lopes et al., 2020). Does the model learn to utilize context? Yes. We answer this question by studying the impact of incorrect context on our model. We replace the correct source context with some random audio segments from the same document, and randomly select the target context from previous translations during decoding. Intuitively, the performance of our model should be intact if it ignores the context. Note that we trained our model with correct contexts but test it with random contexts here. Results in Table 2 show that the randomized context, either sourceor target-side, hurts the performance of our model in both BLEU and APT, similar to the findings in (Voita et al., 2018), and the translation of pronouns suffers more ( > -1.6 APT). Compared to SWBD, the incorrect context has more negative impact on IMED, resulting in worse performance than Baseline (Table 1), although IMED also uses sentence-level translation. We ascribe this to the target prefix constraint in IMED which makes translation errors at early decoding much easier to propagate. We observe that the incorrect target context acts similarly to its source counterpart under IMED, albeit its selection scope is much smaller (only limited to the translated segments), and combining both contexts leads to a slight but consistent performance degradation. These results demonstrate that our model indeed learns to use contextual information for translation. Figure 3 : Case-sensitive tokenized BLEU (top) and APT (bottom) as a function of context size C on MuST-C En-De test set. Figure 4 : Case-sensitive tokenized BLEU (left y-axis) and APT (right y-axis) on MuST-C En-De test set when varying  for IMED. Solid and dashed curves are for BLEU and APT, respectively. C = 2 . How much context sentences should we use? Although adding extra context provides more information, it makes learning harder: neural models often struggle with long sequences. Figure 3 shows the impact of context size on translation. We find that our models do not benefit from context size beyond 2 previous segments. Figure 3 also shows that the overall trend of the impact of C on BLEU and APT is similar for different decoding methods. Increasing C to 1 delivers the best APT, while context-aware ST achieves its best BLEU at C = 2 . We use C = 2 for the following experiments. Impact of  on IMED. IMED heavily relies on the hyperparameter  (Eq. 2) to control its preference between sentence-level and document-level decoding. Figure 4 shows its impact on translation Model ACC hp Baseline (ST+AFS) 48.93 Ours + SWBD 49.90 Ours + IMED 49.66 Ours + IMED  = 1 . Table 3 : Translation accuracy of homophones (ACC hp ) MuST-C En-De test set. C = 2 ,  = 0 . 5 . quality, which clearly reveals a trade-off. The performance of IMED (BLEU and APT) reaches its peak at  = 0 . 4 , and decreases when  becomes either smaller or larger. The optimal value of  for IMED might vary greatly across different language pairs. It also shows some difference across evaluation sets (see Figure 7 in Appendix). In the following experiments, we will apply equal weighting (  = 0 . 5 ), a common choice for model ensembles and not substantially worse than the optimum on this dataset. Impact of context on homophone translation. Homophones (words that sound the same but hold different meanings, such as  I  vs.  eye  and  would  vs.  wood ) and other acoustically similar words increase the learning difficulty of ST models compared to textual MT. To allow for automatic quantitative evaluation, we extract words from the MuST-C test set transcriptions which share the same phonemes with Montreal Forced Aligner (McAuliffe et al., 2017). We collect all homophones and evaluate their translation accuracy (ACC hp ) in the same way as APT. Table 3 shows that context-aware ST outperforms Baseline by > 0.73 ACC hp , where SWBD performs slightly better than IMED. After removing the document-level decoding, IMED (  = 1 . 0 ) performance drops greatly, even underperforming Baseline. While we see some improvements to homophone translations, they are in the same relative range as general improvements from context. Anecdotal examples from manual inspection (see Table 7 in Appendix) indicate that context may at times help disambiguate acoustically similar forms, but that (near-)homophones still remain a salient source of translation errors. Context improves the robustness of ST models to audio segmentation errors. In MuST-C, the audio is already well-segmented, with each segment corresponding to a short transcript. Nevertheless, natural audio, streaming speeches in particular, has no such segment boundaries, and how to parti-Model Random Gold Baseline (ST+AFS) 20.40 27.40 Ours + SWBD 21.83 28.02 Ours + IMED 22.03 28.03 Table 4 : Document -level case-sensitive tokenized BLEU for different models on MuST-C En-De test set with erroneous audio segmentation. We report average BLEU over three runs; each run uses a different random seed to simulate segmentation errors. C = 2 ,  = 0 . 5 . Random/Gold : document-based BLEU when the random/gold segments are used. tion audio itself is an active research area (Rangara-jan Sridhar et al., 2013; Zhang and Zhang, 2020). Since ST models are often trained with gold segments, they inevitably suffer from segmentation errors at inference when the gold ones are unavailable. The bottleneck mainly comes from the incompleteness of each segment, which, we argue, contextual information could alleviate. We simulate segmentation errors by randomly re-segmenting the audio in MuST-C En-De test set based on the given segment number. Especially, given an audio with N gold segments, we randomly re-segment it into N disjoint pieces, where each piece usually has different boundaries against its gold counterpart. 7 We evaluate different ST models with document-based BLEU. Table 4 summarizes the results. Segmentation noise deteriorates translation quality for all ST models to a large degree ( > -6 BLEU). Compared to sentence-level ST, context-aware ST is less sensitive to those errors. In particular, our model with IMED yields a document-based BLEU of 22.03, substantially outperforming Baseline (by 1.63 BLEU). Our results also confirm the findings of Gaido et al. (2020). Context benefits simultaneous translation. Simultaneous translation requires that we start decoding before receiving the whole audio input to minimize latency; operating on such short units increases ambiguity, and the model may be forced to predict future input to account for word order differences, which we hypothesize is easier with access to super-sentential context. We focus on segment-7 Note we intentionally keep the same segment number, N , in the simulated noisy segmentation, because this offers us a fair setup to analyze the impact of segmentation errors on the final translation when compared to the gold segmentation. This avoids the potential influence resulting from mismatched segment number. We leave the study of the model's robustness to genuine segmentation noises to future work. Table 6 : Simultaneous translation results (BLEU, DAL and NE) for different models on MuST-C En-De test set. C = 2 ,  = 0 . 5 . level E2E simultaneous translation, and adopt the re-translation method (Niehues et al., 2016; Arivazhagan et al., 2020b,a) where we translate the source input segment from scratch after every 1 second. For training, we finetune each model for extra 20K steps with a 1:1 mix of full-segment and prefix pairs, following Arivazhagan et al. (2020a). We construct the prefix pairs by uniformly selecting an audio prefix length and then proportionally deciding the target prefix length based on the sentence length. Note that the context inputs in our model are still full segments/sentences. We adopt tokenized BLEU, differentiable average lagging (DAL), and normalized erasure (NE) to evaluate the translation quality, latency and stability, respectively, following Arivazhagan et al. (2020a). Note DAL and NE are measured based on words. Results in Table 6 show that context-aware ST improves translation quality ( > +0.84 BLEU) and reduces translation latency ( > -0.06 DAL) regardless of the decoding method. It also enhances translation stability when the target prefix constraint is applied ( > -0.08 NE, SWBD-Cons & IMED). SWBD performs worse in NE, because it allows changes in the translation of context which increases instability. Overall, context provides extra information to the translation model, before the 0 . Figure 5 : DAL (left y-axis) and NE (right y-axis) as a function of  for IMED on MuST-C En-De test set in simultaneous translation setting. Solid and dashed curves are for DAL and NE, respectively. C = 2 . 0 . 0 : document-level decoding;   1 . 0 : sentence-level decoding. simultaneous translation. Figure 5 further illustrates how context impacts simultaneous translation. With the increase of sentence-level decoding (   1 . 0 ), IMED produces higher DAL and NE, i.e. worse quality. We ascribe the reduction of latency and stability in our model to the inclusion of contextual information. Table 5 summarizes the results for all 8 translation pairs covered by MuST-C. Overall, our model obtains improvements over most metrics and language pairs, despite their different language characteristics. Out of 8 languages, our model performs relatively worse on Es and It with smaller BLEU gains and even negative results in ACC hp . By contrast, our model yields the largest improvement on Ro. In particular, our model with IMED achieves a detokenized BLEU of 23.6 on En-Ro, surpassing the state-of-the-art result 22.2 (Zhao et al., 2020) reported so far. Our experiments confirm the effectiveness of context-aware modeling for end-to-end speech translation. With concatenation-based contextual modeling and appropriate decoding method, we observe positive impact of context on translation. Context-aware ST improves general translation quality in BLEU, and also helps pronoun and homophone translation. ST models become less sensitive to (artificial) audio segmentation errors with context. In addition, context also improves simultaneous translation by reducing latency and erasure. We observe overall positive results over different languages and evaluation metrics on the MuST-C corpus. In the future, we will investigate more dedicated neural architectures to handle long-form speech input. While we relied on a dataset with sentence segmentation in this work, we are interested in removing the reliance on segmentation at inference time to implement the full-fledged streaming translation scenario. We thank the reviewers for their insightful comments. This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreements 825460 (ELITR). Rico Sennrich acknowledges support of the Swiss National Science Foundation (MUTAMUR; no. 176727).
abstain method objective objective abstain abstain abstain other abstain method abstain abstain objective abstain abstain abstain objective abstain abstain abstain result objective method result abstain method method method method objective abstain objective objective objective abstain other other abstain other other other abstain abstain other objective abstain other abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain method abstain abstain method method method method abstain method method method method method abstain method abstain method method method method method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain result abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain result method result abstain abstain abstain result objective method other other other