How to find proper moments to generate partial sentence translation given a streaming speech input? Existing approaches waiting-and-translating for a fixed duration often break the acoustic units in speech, since the boundaries between acoustic units in speech are not even. In this paper, we propose MoSST, a simple yet effective method for translating streaming speech content. Given a usually long speech sequence, we develop an efficient mo notonic s egmentation module inside an encoder-decoder model to accumulate acoustic information incrementally and detect proper speech unit boundaries for the input in s peech t ranslation task. Experiments on multiple translation directions of the MuST-C dataset show that MoSST outperforms existing methods and achieves the best trade-off between translation quality (BLEU) and latency. Our code is available at https://github. com/dqqcasia/mosst . Speech translation (ST) aims at translating from source language speech into target language text, which is widely helpful in various scenarios such as conference speeches, business meetings, cross-border customer service, and overseas travel. There are two kinds of application scenarios, including the non-streaming translation and the streaming one. The non-streaming models can listen to the complete utterances at one time and then generate the translation afterward. While, the streaming models need to balance the latency and quality and generate translations based on the partial utterance, as shown in Figure 1. Recently, end-to-end approaches have achieved remarkable progress in non-streaming ST. Previous work (Weiss et al., 2017; Brard et al., 2018; Livescu and Goldwater, 2019; Bansal et al., 2019;  Equal contribution.  Work is done while at ByteDance. I remember my first fire. Ich erinnere mich an mein erstes Feuer. Listen Write Figure 1: An illustration of streaming speech-to-text translation. ST models listen to the audio in source language, and generate tokens in target language. Alinejad and Sarkar, 2020; Stoian et al., 2020) Ansari et al. (2020) has shown that an end-to-end model achieves even better performance compared to the cascaded competitors. However, attempts at end-to-end streaming ST are still not fully explored. Traditional streaming ST is usually formed by cascading a streaming speech recognition module with a streaming machine translation module (Oda et al., 2014; Dalvi et al., 2018). Most of the previous work focuses on simultaneous text translation (Gu et al., 2017a). Ma et al. (2019a) propose a novel waitk strategy based on the prefix-to-prefix framework, which is one of the popular research methods of simultaneous text translation. For end-to-end streaming ST, Ma et al. (2020b); Ren et al. (2020); Ma et al. (2021a) introduce the methodology of streaming machine translation into streaming ST and formalize the task, which belongs to the first study to propose simultaneous ST in an end-to-end manner. However, those previous streaming ST systems generally treat a fixed time-span of audio as a acoustic unit and translate new words based on fixed time segmentation, which might be unfavorable for streaming ST translation. Since the speaker's speech speed and the length of the phonemes are 680 Audio Waveform Self Attention FeedForward x N AcousticEncoder Monotonic SegmentationModule Self Attention FeedForward x N CrossAttention Translation Transcription CE loss LP loss Figure 2: Overview of the proposed MoSST. distinct, previous methods cannot find the best policy to tell whether to continue reading source audio or translate new words when the source audio is streaming in. Hence, we expect the model can determine whether the streaming audio information input is enough to translate new words, similar to the manual simultaneous interpretation. This idea inspires Monotonic-segmented Streaming Speech Translation (MoSST) system. Specifically, we design a new module that helps to judge the acoustic boundaries of the input audio. We then propose a translation strategy that enables the model to decide whether to read the audio stream or write new tokens given the audio prefix. With the new module and decoding strategy, the model's performance on streaming speech translation has been significantly improved. We highlight our innovations and findings as follows:  We propose a simple but effective framework, MoSST for streaming speech translation. We introduce a new monotonic segmentation module to segment audio waveform into acoustic units, based on which we design the adaptive decision strategy which dynamically decides when to translate a new word in streaming scenarios. We validate MoSST on the MuST-C dataset. The results show that our model significantly outperforms SOTA baselines. Surprisingly, we also find that MoSST can rival or even surpass other SOTA systems in non-streaming speech translation. Furthermore, we conduct a comprehensive study to analyze the utility of the proposed module and decoding strategy. This section first formulates the ST task in streaming and non-streaming scenarios. Then, we introduce the detailed architecture of MoSST, as shown in Figure 2. Finally, we give the training and inference strategies of MoSST for streaming and non-streaming cases. The ST corpus usually contains speech-transcription-translation triples ( x , z , y ) . Specially, x = ( x 1 , ..., x T x ) is a sequence of acoustic features. z = ( z 1 , ..., z T z ) and y = ( y 1 , ..., y T y ) represents the corresponding transcription in source language and the translation in target language respectively. Usually, the acoustic feature x is much longer than text sequences z and y , as the sampling rate of audio is usually above 16,000 Hz, and each word syllable (about 300 ms) will be recorded by thousands of sampling points. The streaming ST model aims to translate instantly when speech audio streams in, that is, given a valid audio prefix x < , where  is the time span of the audio piece, we expect the model can translate enough information y <K , where K is the maxi-681 mum number of tokens that the model can translate as time  , i . where  is the parameters of the streaming ST model. Our goal is to find the best   that maximizes the Pr( y <K | x < ) in Eq. 1. Note that in our research scenario, we require that the translated piece of the sentence shall not be modified once generated, similar to the settings in simultaneous machine translation (Ma et al., 2019a). monotonic segmentation, and a standard Transformer. Acoustic Encoder The conventional acoustic encoder using FBANK (log-Mel filterbank, FBANK) as feature extractors faces reduced performance with insufficient training data (San et al., 2021), which is especially the case in speech-to-text translation tasks. The FBANK also leads to potential information loss, and may corrupt long-term correlations (Pardede et al., 2019). To tackle such problems, we apply the recently-proposed pre-trained acoustic models (Chen et al., 2020; Baevski et al., 2020) as the feature extractor for MoSST. Those pre-trained acoustic models learn the speech representation in a self-supervised learning (SSL) way. Since pre-trained acoustic models require only a large amount of unlabeled speech, which also alleviates the corpus shortage of ST tasks. In this paper, we utilize Wav2Vec2 (Baevski et al., 2020) as our instance. Monotonic Segmentation Module The previous speech translation model generally attends the whole audio sequence to the translation tokens with a sequence-to-sequence (seq2seq) framework, which brings two problems: 1) the model does not learn the alignment between audio and translation explicitly, which may confuse the streaming translation model on whether it has read enough acoustic information when generating the translated text; 2) the audio sequences are usually much longer than text sequences, which is computationally demanding for the conventional encoder-decoder speech-to-text model to apply the global attention mechanism. Such high computational cost deviates from the requirements in streaming translation scenarios. We introduce a Monotonic Segmentation Module (MSM), to relieve drawbacks of existing models. The MSM is inspired by the integrate-and-fire (IF) model (Abbott, 1999; Dong and Xu, 2020; Yi et al., 2021). Specifically, IF neuron has two modes: integrate and firing . In integrate mode, the IF neuron dynamically receives signals and accumulates information; when the received information exceeds a certain threshold, IF neuron enters firing mode, at which time it outputs a signal (a.k.a. spiking), where the accumulated state contains information received in the previous integrate phase; and finally, the IF neuron will reset itself and reenter the integrate mode once the firing mode ends. In the MSM, we utilize the integrate-and-fire cycle to dynamically locate the boundaries of meaningful speech segments. At the integrate mode, the model keeps reading and processing speech frames, while at firing mode the model writes the translated tokens. MSM takes the representation from the Acoustic Encoder and uses one of the dimensions as signals for integrate-and-fire. These signals are passed through a Sigmoid function to produce integration weights. Once the weights are accumulated to a certain threshold (e.g. =1.0), the module marks the boundary of the current segment and enters a firing mode. It then aggregates the rest dimensions of encoder representations according to the weights within this segment. These are passed to further processing blocks for WRITE operation. Where h is the acoustic vector as an output of the acoustic encoder, and its subscript denotes the scale value of h at timestamp t and d -th dimension ( i . e ., we use the last dimension as the input of IF neu-rons). The Sigmoid value of the scale h t,d is the current weight, denoted as  t . We use the current weight to decide mode conversion from integrate to firing: when the accumulated sum of  t exceeds the threshold value, the model is believed to have READ sufficient speech signals in this integrate stage, and the IF neural fires the accumulated in-682 formation l = ( l 1 , ..., l u ) to fulfill one integrate-and-fire cycle. And S u represents the firing step corresponding to l u . Note that the accumulated information l is calculated as a weighted sum of acoustic vectors h t at a single integrate stage t . We call it as information weight  (cid:48) t , which helps to scale the amount of information contained in each integrate stage. We calculate the information weight  (cid:48) t by normalizing the current weight  t with the number of tokens in the corresponding transcription n  , which divides the length of the accumulated acoustic vector  n . Transformer block The last module of the MoSST is the standard Transformer. The Transformer blocks take the integrated acoustic vector l from the MSM layer as the input, which aims to extract the semantic feature ( h SE ) of the input audio. Since MSM has significantly compressed the length of acoustic features, the Transformer can attend the input and output directly without the excessive computational overhead. Note that to ensure that MSM learns the correct length of acoustic units, we use the length of the corresponding transcription as a supervised signal and introduce length penalty loss (LP loss in Figure 2) to assist MSM's learning. During inference, an extra rounding operation is applied on  n to simulate n  . Based on the matched sequence length, the accumulated acoustic vector l is mapped back into the model size by a randomly initialized fully connected layer. Multi-task Joint Training with ASR MoSST jointly fulfills the ST and ASR tasks with the multitask learning (MTL) strategy as its main model. To distinguish two tasks, we add two special task indicators at the beginning of the text as the BOS operator for decoding. For example, if the audio input for " Thank you . " is in English, for ASR, we use [en] as the BOS and decode z = " [en] Thank you . ". We add [De] at the start of German translation, thus y is " [De] Danke . " Both ST and ASR are optimized with cross-entropy (CE loss in Figure 2) losses, defined in Equation (7) and (8) respectively. where the decoder probability p  is calculated from the final softmax layer based on the output of the decoder. We use the joint training strategy to optimize all modules. The overall objective function is the weighted sum for all aforementioned losses: L (  ; x , y , z ) =  L lp (  ; x , z ) +  L ce (9) Where L ce represents L asr or L st . In the following experimental sections,  is set to 0.05, and  is set to 1 by default. Waitk Policy MoSST adopts waitk policy for streaming translation, which originates from simultaneous machine translation (Ma et al., 2019a). Waitk policy waits for K source tokens and then translates target tokens concurrently with the source streams in ( i . e ., output N tokens when given N + K source tokens). The previous online ST systems adopt Pre-fix Decision (Ma et al., 2021b, 2020b) for waitk policy, where a fixed time span (usually 280ms) of the source waveform is regarded as a new unit. However, the pre-fixed decision is limited on real-world scenarios since the speech speed of speakers and the length of acoustic units are distinct, where a fixed time stride guarantees neither sufficient information if the phonemes are too long, nor a proper translation latency if the phonemes are too short. Adaptive Decision We propose a new decision strategy for streaming speech translation, namely Adaptive Decision . Our new strategy dynamically decides when to write the new token according to the integrated state length of MSM ( i . e ., | l u | in Equation ( 5) ). Since MSM scales up the acoustic information monotonically, the model can estimate the acoustic boundary for each units in the audio. We use such integrate feature as a basis to tell whether the information carried by the waveform segment is sufficient; hence the proposed adaptive decision revises the drawbacks in fixed decision. We propose our new decoding policy in Algorithm 1. The new policy utilizes waitk to decide when to write new translation tokens and adaptive 683 Algorithm 1: Adaptive Decision Strategy Input: The waveform sequence x , the MSM model M , wait lagging K Output: The translated sentence  y 1 initialization: the read waveform segment  x = [] , the output sentence  y = [] ; 2 while  y i  1 is not EndOfSentence do 3 calculate MSM integrated state l u ; 4 if  x == x ; 5 then /* the waveform is finished */ /* write new token */ 6  y =  y + decoder.predict(); 7 M.decoder.update(  y ) ; 8 else if | l u |  |  y | < K ; 9 then /* read waveform */ 10  x =  x + new_segment ( x ) ; 11 M.encoder.update(  x ) 12 else /* write new token */ 13  y =  y + decoder.predict(); 14 M.decoder.update(  y ) ; 15 end 16 return  y ; decisions to decide how long the input is regarded as a unit. Specifically, during the online ST translation, the model shall decide whether to read new audio frames or translate a new word at any time, called the READ/WRITE decision. We denote  x as the audio sub-sequence that the model has READ from the source and  y as the sentence prefix that has been already generated. The waitk policy makes the READ/WRITE decision according to the length difference between the MSM integrated state | l u | and the generated sentence |  y | . When the integrated state | l u | is K word behind the generated |  y | , the MoSST generates a new token (line 12) and updates decoder states recursively, otherwise, the model waits and reads the audio streaming (line 9), and updates the encoder states. Train-full Test-k Streaming translation needs to predict the output based on part of the input. If the train-full test-k paradigm is applied, the streaming performance will decrease a lot due to the mismatch between training and inferring. The previous streaming work generally uses a prefix-to-prefix training framework (Ma et al., 2019a), implemented by a unidirectional encoder and decoder, and equipped with the waikk policy. In MoSST, the learned monotonic segmentation module allows our model to have streaming decoding capability without performance drop. MuST-C 1 (Di Gangi et al., 2019a) is a multilingual ST corpus with triplet data sources: source audio, transcripts, and text translations. To the best of our knowledge, MuST-C is currently the largest ST dataset available. It includes data from English TED talks with auto-aligned transcripts and translations at the sentence level. We mainly conduct experiments on English-German and English-French language pairs. And we use the dev and tst-COMMON sets as our development and test data, respectively. For speech input, the 16-bit raw wave sequences are normalized by a factor of 2 15 to the range of [  1 , 1) . For text input, on each translation pair, all texts (including transcript and translation) are preprocessed in the same way. Texts are case-sensitive. We keep and normalize the punctuations, but remove non-print characters. We tokenize sentences with Moses tokenizer 2 and filter out samples longer than 250 words. For subword modeling, we use a unigram sentencepiece (Kudo and Richardson, 2018) with a dictionary size of 10000. On each translation direction, the sentencepiece model is learned on all text data from ST corpora. Model Configuration For audio input, the Wav2Vec2 module follows the base 3 configuration in Baevski et al. (2020). It uses parameters self-supervised pre-trained on LibriSpeech audio data only. The subsequently shared Transformer module has a hidden dimension of 768 and 4 attention heads. The encoder is 8 layers, and the decoder 1 https://ict.fbk.eu/must-c/ 2 https://github.com/moses-smt/ mosesdecoder/blob/master/scripts/tokenizer/tokenizer.perl 3 https://dl.fbaipublicfiles.com/ fairseq/wav2vec/wav2vec_small.pt 684 is 6 layers. We use the simplified version of the Continuous IF implementation (Yi et al., 2021) for MSM module, which introduces no additional parameters except for a fully connected layer. Experimental Configuration We use an Adam optimizer with  1 = 0 . 9 ,  2 = 0 . 98 , and 4k warmup updates. We set the maximum training batch of the waveform audio token to be 3.2 million. We apply an inverse square root schedule algorithm for the learning rate. We average 10 consecutive checkpoints around the one with the best dev loss and adopt a beam size of 5. We implement our models in Fairseq (Ott et al., 2019). For offline translation, the model's performance is mainly evaluated with quality metrics. While for streaming translation, ST model is evaluated by the latency-quality trade-off curves. Quality Metrics We quantify translation accuracy with detokenized BLEU (Papineni et al., 2002) using sacreBLEU 5 . Latency Metrics Existing simultaneous translation work mainly focuses on the latency evaluation of text translation, and has proposed computation unaware metrics, such as Average Proportion (AP) (Cho and Esipova, 2016), Average Latency (AL) (Ma et al., 2019a), Continues Wait Length (CW) (Gu et al., 2017b) and Differentiable Average Lagging (DAL) (Cherry and Foster, 2019). Ma et al. (2020a) extends the latency metrics of text translation into ST, including AL, AP, and DAL. The latency metrics for streaming MoSST are evaluated by AL, DAL, and AP based on the SimulEval toolkit 6 (Ma et al., 2020a). We compare the performance of our method with published work on streaming ST tasks. SimulST (Ma et al., 2020b) introduces the wait-k training strategy in simultaneous text translation into simultaneous ST tasks. The comparison result on MuST-C EN-DE tst-COMMON set is shown in Figure 3. It can be seen that MoSST is significantly better than the baseline system in 4 https://github.com/pytorch/fairseq/ blob/main/examples/speech_to_text/docs/simulst_mustc_example.md 5 https://github.com/mjpost/sacrebleu 6 https://github.com/facebookresearch/ SimulEval all the three latency metrics and the quality metric. SimulSpeech (Ren et al., 2020) also adopts the waitk strategy and leverages the connection-ist temporal classification (CTC) decoding to split the input streaming speech chunk in real-time. Besides, SimulSpeech introduces attention-level and data-level knowledge distillation (KD) to improve performance. The comparison result on MuST-C EN-DE tst-COMMON set is shown in Table 1. It can be seen that when k ranges from 1 to infinite, our method significantly outperforms SimulSpeech. Existing work all uses the waitk training strategy implemented with a unidirectional mask, which would damage the performance of offline evaluation in full context. While MoSST can serve well both non-streaming and streaming translation. At the same time, the shrinking mechanism based on the MSM can speed up model convergence, which we give a detailed analysis in Sec A.2.2 of the Appendix. We also compare the performance of our method with published work on offline ST tasks under experimental settings without external supervised training data. The result is shown in Table 2. Fairseq (Wang et al., 2020a), ESPnet (Inaguma et al., 2020), and NeurST (Zhao et al., 2020) are recently emerging R&D toolkits for ST. Transformer ST uses a standard SpeechTransformer (Dong et al., 2018) model structure, with a pre-trained ASR model to initialize the encoder and a pretrained MT model to initialize the decoder. Zhang et al. (2020a) propose adaptive feature selection (AFS) for ST, which applies L 0 DROP (Zhang et al., 2020b) to dynamically estimate the importance of each encoded speech feature. STAST (Liu et al., 2020b) uses a speech-to-text adaptation method to bridge the modality gap in the semantic space by MTL and representation regulation with MT. Le et al. (2020) adapt the dual-decoder transformer with a dual-attention mechanism to joint ASR and ST for both bilingual (BL) and multilingual (ML) settings. Compared with the best results published so far, MoSST can achieve an improvement of 1.3 BLEU and 0.7 BLEU respectively. It should be noted that the previous methods can be integrated into MoSST to expect better performance. We will leave it for further exploration. We conduct ablation studies to demonstrate the effectiveness of the design of MoSST, including the monotonic segmentation module, multi-task joint training with ASR, and self-supervised acoustic representation. The ablation study results can be seen in Table 3. The translation quality decreases 1000 1500 2000 2500 3000 3500 4000 Differentiable Average Lagging 5 10 15 20 BLEU Stride=280ms Stride=320ms Stride=400ms Stride=480ms Figure 4: The translation quality against the latency metrics (DAL) on the tst-COMMON set of MuST-C En-De dataset. significantly when each of the modules or strategies is emitted successively. The self-supervised acoustic representation can bring almost 2 BLEU on both EN-DE and EN-FR datasets, which shows that large-scale SSL brings hope to solving the data scarcity problem of end-to-end ST. For EN-DE language pair, joint training with the auxiliary ASR task has a performance gain of 0.8 BLEU. And the monotonic segmentation module has an additional 2.2 performance gain to our method. The results show a consistent performance improvement on EN-FR language pair. This verifies the outstanding advantage of the monotonic soft attention mechanism of MSM in extracting contextual acoustic representations. For the pre-fixed decision decoding strategy, the parameter setting of stride is very important. In Fig-686 1000 1500 2000 2500 3000 Differentiable Average Lagging 5 10 15 20 BLEU pre-fix decision adaptive decision Figure 5: The translation quality against the latency metrics (DAL) on the tst-COMMON set of MuST-C En-De dataset. ure 4, we compare the influence of different strides on the pre-fixed decision strategy. It can be seen that increasing stride within a certain range will have a positive impact on the latency-bleu trade-off. But the model also tends to fall into the field of a larger latency. We have proposed an adaptive decision in Section 2.4. To better emphasize the latency factor, we compare the performance of the adaptive decision and the pre-fixed decision on the tst-COMMON test subset of MuST-C EN-DE. The results are shown in Figure 5. Compared with the pre-fixed strategy decoding method, the adaptive strategy decoding method has a better balance between delay and quality. Through observation, it is found that the adaptive strategy can ignore the silent frames. For example, after predicting a punctuation, it will read continuously to accumulate enough source acoustic information. In addition, the adaptive strategy can further reduce the delay by setting the number of WRITE operations after the accumulated information is sufficient according to the length ratio of the source sentences and the target sentences between different language pairs, which requires further exploration. In Figure 6, we show the ground truth alignment and the predicted firing positions learned by MoSST. We can see that what MSM learned is the acoustic boundary, not to mimic waitk . Therefore, the length of the audio chunk can be adaptively read in during streaming decoding, while ensuring that each chunk includes a complete acoustic unit. Speech Translation Brard et al. (2016) have given the first proof of the potential for end-to-end speech-to-text translation without using the intermediate transcription. The training method based on pre-training (Weiss et al., 2017; Brard et al., 2018; Livescu and Goldwater, 2019; Bansal et al., 2019; Alinejad and Sarkar, 2020; Stoian et al., 2020; Dong et al., 2021a) can effectively use pre-trained models with better performance as initialization to speed up the convergence of the ST model. Multi-task learning (Weiss et al., 2017; Brard et al., 2018; Liu et al., 2020a; Indurthi et al., 2020; Han et al., 2021; Ye et al., 2021) can fully optimize the model parameters and improve the performance with the aid of auxiliary tasks. Knowledge distillation has been proved to be efficient to learn from pre-trained models (Liu et al., 2019, 2020b; Dong et al., 2021b). Le et al. (2021) introduce adapter for multilingual speech translation. Similarly, Kano et al. (2017); Wang et al. (2020b) introduce curriculum learning methods, including different learning courses of increasing difficulty. To overcome data scarcity, Jia et al. (2019); Pino et al. (2019) augment data with pseudo-label generation, and Bahar et al. (2019); Di Gangi et al. (2019b); McCarthy et al. (2020) introduce noise-based spectrum feature enhancement. Zhang et al. (2020a) propose adaptive feature selection to eliminate uninformative features and improve performance. Streaming Speech Translation Traditional streaming ST is usually formed by cascading a streaming ASR module and a streaming machine translation module (Oda et al., 2014; Dalvi et al., 2018). The ASR system continuously segments and recognizes the transcription of the audio segment, and then the machine translation system continuously translates the text segment output from the upstream. Most of the previous work focuses on simultaneous text translation (Gu et al., 2017a). Gu et al. (2017a) learn an agent to decide when to read or write. Ma et al. (2019a) propose a novel waitk strategy based on the prefix-to-prefix framework to synchronize output after reading k history tokens. Many following work propose some improvement strategies based on adaptive waitk (Zheng et al., 2019; Zhang et al., 2020c; Zhang and Zhang, 2020) and efficient decoding (Elbayad et al., 2020; Zheng et al., 2020). Some monotonic attention methods (Arivazhagan et al., 2019; Ma et al., 2019b; 687 0.6 0.4 0.2 0.0 0.2 0.4 0.6 s p ee c h i remember my first fire 0.00 0.05 0.10 0.15 0.20 0.25 a l p h a 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Time 0.0 0.2 0.4 0.6 0.8 1.0 1.2 a cc u m u l a t i o n Figure 6: An example speech and its corresponding learned firing positions by MoSST. Schneider and Waibel, 2020) have been proposed to model the monotonic alignment of input and output. Arivazhagan et al. (2020a,b) propose a retranslation strategy, allowing the model to modify the decoding history to improve the performance of streaming translation. Ma et al. (2020b) propose SimulST, which applies the waitk method from streaming machine translation (Ma et al., 2019a) into streaming ST. Ren et al. (2020) propose SimulSpeech, which uses knowledge distillation to guide the training of the streaming model and the con-nectionist temporal classification (CTC) decoding to segment the audio stream in real-time. Ma et al. (2021a) enable the streaming model to handle long input by equipping with an augmented memory encoder. Chen et al. (2021) use a separate and synchronized ASR decoder to guide the ST decoding policy. Zeng et al. (2021b) introduce a blank penalty to enhance performance in simultaneous scenarios. We propose MoSST, a simple and effective framework for online speech-to-text translation. MoSST consists of a pretrained acoustic model, a monotonic segmentation module, and a standard Transformer, along with the multitask training strategy and the adaptive decision strategy. The monotonic segmentation module and the adaptive decision strategy tell our method when to translate. Moreover, the pre-trained acoustic encoder and the multitask training strategy boost our method's ability to predict what to generate. The experiment on MUST-C datasets validates the effectiveness of MoSST over previous work. The results show that MoSST can achieve a better trade-off between quality and latency over prior end-to-end models and cascaded models in diverse latency settings. Besides, we also find MoSST can rival non-streaming speech translation SOTA systems given the complete audio waveform.
abstain abstain objective objective abstain other abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain objective abstain objective objective abstain objective objective abstain result result objective abstain method method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain method method method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain other abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other other other other other other other other other other other other other other other other other other other objective abstain abstain method abstain abstain result