Five years after the first published proofs of concept, direct approaches to speech translation (ST) are now competing with traditional cascade solutions. In light of this steady progress, can we claim that the performance gap between the two is closed? Starting from this question, we present a systematic comparison between state-of-the-art systems representative of the two paradigms. Focusing on three language directions (English German/Italian/Spanish), we conduct automatic and manual evaluations, exploiting high-quality professional post-edits and annotations. Our multi-faceted analysis on one of the few publicly available ST benchmarks attests for the first time that: i) the gap between the two paradigms is now closed, and ii) the subtle differences observed in their behavior are not sufficient for humans neither to distinguish them nor to prefer one over the other. Speech translation (ST) is the task of automatically translating a speech signal in a given language into a text in another language. Research on ST dates back to the late eighties and its evolution followed the development of the closely related fields of speech recognition (ASR) and machine translation (MT) that, since the very beginning, provided the main pillars for building the so-called cascade architectures. With the advent of deep learning, the neural networks widely used in ASR and MT have been adapted to develop a new direct ST paradigm. This approach aims to overcome known limitations of the cascade one (e.g. architectural complexity, error propagation) with a single encoder-decoder architecture that directly translates the source signal bypassing intermediate representations. Until now, the consolidated underlying technologies and the richness of available data have upheld the supremacy of cascade solutions in industrial applications. However, architectural simplicity, reduced information loss and error propagation are the ace up the sleeve of the direct approach, which has rapidly gained popularity within the research community in spite of the critical bottleneck represented by data paucity. Within a few years after the first proofs of concept (Berard et al., 2016; Weiss et al., 2017), the performance gap between the two paradigms has gradually decreased. This trend is mirrored by the findings of the International Workshop on Spoken Language Translation (IWSLT), 1 a yearly evaluation campaign where direct systems made their first appearance in 2018. On English-German, for instance, the BLEU difference between the best cascade and direct models dropped from 7.4 points in 2018 (Niehues et al., 2018) to 1.6 points in 2019 (Niehues et al., 2019b). In 2020, participants were allowed to choose between processing a pre-segmented version of the test set or the one produced by their own segmentation algorithm. As reported in (Ansari et al., 2020), the distance between the two paradigms further decreased to 1.0 BLEU point in the first condition and, for the first time, it was slightly in favor of the best direct model in the second condition, with a small but nonetheless meaningful 0.24 difference. So, quoting Ansari et al. (2020), is the cascade solution still the dominant technology in ST? Has the direct approach closed the huge initial performance gap? Are there systematic differences in the outputs of the two technologies? Are they distinguishable? Answering these questions is more than running an evaluation exercise. It implies pushing research towards a deeper investigation of direct 1 http://iwslt.org ST, finding a path towards its wider adoption in industrial settings and motivating higher engagement in data exploitation and resource creation to train the data-hungry end-to-end neural systems. For all these reasons, while Ansari et al. (2020) were cautious in drawing firm conclusions, in this paper we delve deeper into the problem with the first thorough comparison between the two paradigms. Working on three language directions (ende/es/it), we train state-of-the-art cascade and direct models (  3), running them on test data drawn from the MuST-C corpus (Cattoni et al., 2020). Systems' behavior is analysed from different perspectives, by exploiting high-quality post-edits and annotations by professionals. After discussing overall systems' performance (  4), we move to more fine-grained automatic and manual analyses covering two main aspects: the relation between systems' performance and specific characteristics of the input audio (  5), and the possible differences in terms of lexical, morphological and word ordering errors (  6). We finally explore whether, due to latent characteristics overlooked by all previous investigations, the output of cascade and direct systems can be distinguished either by a human or by an automatic classifier (  7). Together with a comparative study attesting the parity of the two paradigms on our test data, another contribution of this paper is the release of the manual post-edits that rendered our investigation possible. The data is available at: https://ict.fbk.eu/mustc-post-edits . Cascade ST. By concatenating ASR and MT components (Stentiford and Steer, 1988; Waibel et al., 1991), cascade ST architectures represent an intuitive solution to achieve reasonable performance and high adaptability across languages and domains. At the same time, however, they suffer from well-known problems related to the concatenation of multiple systems. First, they require ad-hoc training and maintenance procedures for the ASR and MT modules; second, they suffer from error propagation and from the loss of speech information (e.g. prosody) that might be useful to improve final translations. Research has focused on mitigating error propagation by: i) feeding the MT system with ASR data structures (e.g. ASR n-best, lattices or confusion networks) which are more informative than the 1-best output (Lavie et al., 1996; Matusov et al., 2005; Bertoldi and Federico, 2005; Beck et al., 2019; Sperber et al., 2019), and ii) making the MT robust to ASR errors, for instance by training it on parallel data incorporating real or emulated ASR errors as in (Peitz et al., 2012; Ruiz et al., 2015; Sperber et al., 2017; Cheng et al., 2019; Di Gangi et al., 2019a). Although the former solutions are effective to some extent, state-of-the-art cascade architectures (Pham et al., 2019; Bahar et al., 2020) prefer the latter, as they are simpler to implement and maintain. Direct ST. To overcome the limitations of cascade models, Berard et al. (2016) and Weiss et al. (2017) proposed the first direct solutions bypassing intermediate representations by means of encoder-decoder architectures based on recurrent neural networks. Currently, more effective solutions (Potapczyk and Przybysz, 2020; Bahar et al., 2020; Gaido et al., 2020) rely on ST-oriented adaptations of Transformer (Vaswani et al., 2017) integrating the encoder with: i) convolutional layers to reduce input length, and ii) penalties biasing attention to local context in the encoder self-attention layers (Povey et al., 2018; Sperber et al., 2018; Di Gangi et al., 2019b). Though effective, these architectures have to confront with training data paucity, a critical bottleneck for neural solutions. The problem has been mainly tackled with data augmentation and knowledge transfer techniques. Data augmentation consists in producing artificial training corpora by altering existing datasets or by generating ( audio , translation ) pairs through speech synthesis or MT (Bahar et al., 2019b; Nguyen et al., 2020; Ko et al., 2015; Jia et al., 2019). Knowledge transfer (Gutstein et al., 2008) consists in passing (here to ST) the knowledge learnt by a neural network trained on closely related tasks (here, ASR and MT). Existing ASR models have been used for encoder pre-training (Berard et al., 2018; Bansal et al., 2019; Bahar et al., 2019a) and multi-task learning (Weiss et al., 2017; Anastasopoulos and Chiang, 2018; Indurthi et al., 2020). Existing neural MT models have been used for decoder pre-training (Bahar et al., 2019a; Inaguma et al., 2020), joint learning (Indurthi et al., 2020; Liu et al., 2020) and knowledge distillation (Liu et al., 2019). Previous comparisons. Most of the works on direct ST also evaluate the proposed solutions against a cascade counterpart. The conclusions, however, are discordant. Looking at recent works, Pino et al. (2019) show similar scores, Indurthi et al. (2020) report higher results for their direct model, while Inaguma et al. (2020) end up with the opposite finding. The main problems of these comparisons are that: i) not all the architectures are equally optimized, ii) for the sake of fairness in terms of training data, cascade systems are restricted to unrealistic settings with small training corpora that penalize their performance, and iii) evaluation always relies only on automatic metrics computed on single references. The IWSLT campaigns (Niehues et al., 2019a; Ansari et al., 2020) set up a shared evaluation framework where systems built on a large set of training data are optimized to achieve the best performance, independently from the underlying architecture. In the last round, direct models approached, and in one case (Potapczyk and Przybysz, 2020) outperformed, the cascade ones. However, the evaluation was run only on one language pair, by solely relying on automatic metrics and single references. In this paper, we overcome these limitations by comparing the two paradigms on three language pairs, using different metrics, multiple references (including professional post-edits) as well as fine-grained automatic and manual analysis procedures. To maximize the cross-language comparability of our analyses, we built the cascade and direct ST systems for ende/es/it with the same core technology, based on Transformer. Their good quality is attested by the comparison with the winning system at the IWSLT-20 offline ST task (Bahar et al., 2020), 2 which consists of an ensemble of two cascade models scoring 28.8 BLEU on the en-de portion of the MuST-C Common test set. On the same data, our cascade and direct models achieve similar BLEU scores, respectively 28.9 and 29.1 (see Table 1). 3 On en-es and en-it, identical architectures perform similarly or better (up to 32.9 BLEU on en-es). Although BLEU scores are not strictly comparable across languages, we can safely consider all our models as state-of-the-art. 2 In the pre-segmented data condition (Ansari et al., 2020). 3 Also the ASR performance of our cascade solution (10.2 WER on MuST-C Common ) is in line with the results obtained by Bahar et al. (2020) for their best ASR model. Data. Our evaluation data is drawn from the TED-based MuST-C corpus (Cattoni et al., 2020), the largest freely available multilingual corpus for ST. It covers 14 language directions, with English audio segments automatically aligned with their corresponding manual transcriptions and translations. The ende/es/it MuST-C Common test sets contain the same 27 TED talks, for a total of around 2,500 segments largely overlapping across languages. 4 For all the three language pairs, we selected subsets of MuST-C Common containing the same English audio portions from each talk, in order to obtain representative groups of contiguous segments that are comparable across languages. Furthermore, to ensure high data quality, we manually checked the selected samples and kept only those segments for which the audio-transcript-translation alignment was correct. Each of the three resulting test sets  henceforth PE-sets  is composed of 550 segments, corresponding to about 10,000 English source words. Post-editing. A key element of our multi-faceted analysis is human post-editing (PE), which consists in manually correcting systems' output according to the input (the source audio in our case). In PE-based evaluation, the original output is compared against its post-edited version using distance-based metrics like TER (Snover et al., 2006). This allows for counting only the true errors made by a system, without penalising differences due to linguistic variation as it happens when exploiting independent references. This makes PE-based evaluation one of the most prominent methodologies used for translation quality assessment (Snover et al., 2006, 2009; Denkowski and Lavie, 2010; Cettolo et al., 2013; Bojar et al., 2015; Graham et al., 2016; Bentivogli et al., 2018b). To collect the post-edits for our study, we strictly followed the methodology of the IWSLT 2013-2017 evaluation campaigns (Cettolo et al., 2013), which offered us a consolidated framework and best practices to draw upon. Our cascade and direct systems were both run on the PE-sets to be post-edited. To guarantee high quality post-edits, for each language we hired two professional translators with experience in subtitling and post-editing. Moreover, in order to cope with translators' vari-4 MuST-C Common segments can vary across languages due to the automatic procedures of segmentation, audio-text alignment and filtering that were applied to the talks. ability (i.e. more/less aggressive editing strategies), the outputs of the two ST systems were randomly assigned ensuring that each translator worked on all the 550 segments, post-editing an equal number of outputs from both systems. The task was performed with a CAT tool 5 that displays the manual transcript of the audio together with the ST output to be edited. However, since ST systems take as input an audio signal, we also provided translators with the audio file of each segment, asking them to post-edit strictly according to it. 6 For each language pair, the final PE-set used in our study consists of the 550 MuST-C original audio-transcript-translation triplets plus two additional sets of reference translations, i.e. the post-edited versions of the two systems' outputs. Analyses. The collected post-edits are exploited to assess overall systems' performance (  4) as well as to carry out deeper quantitative and qualitative analyses aimed to shed light on possible systematic differences in systems' behavior (  5.1 and  6.1). Focusing on specific aspects of the ST problem, the inquiry is also performed by means of manual annotation of systems' outputs (  5.2,  6.2 and  7.1). Due to the linguistic nature of this task, centred on fine-grained aspects requiring a variety of skills in both evaluation and ST technology, for such analyses we relied on three researchers in translation technology  one per language pair  with a strong background in linguistics, excellent knowledge of the addressed languages (C2 or native), as well as strong expertise in systems' evaluation. We compute overall performance results both on the PE-sets and on the MuST-C Common test sets. Our primary evaluation is based on the collected post-edits. We consider two TER-based 7 metrics: i) human-targeted TER (HTER) computed between the automatic translation and its human post-edited version, and ii) multi-reference TER (mTER) computed against the closest reference among the three available ones (two post-edits and the official reference from MuST-C). The latter metric better accounts for post-editors' variability, making the evaluation more reliable and informative. For the sake of completeness, in Table 1 we also report Sacre-5 www.matecat.com 6 The ad-hoc ST PE guidelines given to translators are included in Appendix B. 7 www.cs.umd.edu/snover/tercom BLEU 8 (Post, 2018) and TER scores computed only on the official MuST-C Common references. A bird's-eye view of the results shows that, in more than half of the cases, performance differences between cascade and direct systems are not statistically significant. When they are, the raw count of wins for the two approaches is the same (4), attesting their substantial parity. Looking at our primary metrics (HTER and mTER), systems are on par on en-it and en-de, while for en-es the direct approach significantly outperforms the cascade one. This difference, however, does not emerge with the other metrics. Indeed, BLEU and TER scores computed against the official references are less coherent across metrics and test sets. For instance, on the en-it PE-set the cascade system significantly outperforms the direct one in terms of BLEU score, while TER shows the opposite on MuST-C Common . Interestingly, the scores obtained using independent references can also disagree with those computed with post-edits. This is the case of en-es, where significant HTER and mTER reductions attest the superiority of the direct system, while most BLEU and TER scores are still in favor of the cascade. On the one hand, primary evaluation scores suggest that the rapidly advancing direct technology has eventually reached the traditional cascaded approach. On the other, the highlighted incongruities confirm widespread concerns about the reliability of fully automatic metrics  based on independent references  to properly evaluate neural systems (Way, 2018). This calls for deeper quantitative and qualitative analyses. Those presented in the next sections investigate performance differences focusing on two main aspects: the impact of specific input audio properties (  5), and the linguistic errors made by the systems (  6). 8 BLEU+c.mixed+#.1+s.exp+tok.13a+v.1.4.3 5 ST Quality and Audio Properties 5.1 Automatic Analysis The two ST approaches handle the input audio differently: the cascade one by means of a dedicated ASR component that produces intermediate transcripts; the direct one by extracting all the relevant information to translate in an end-to-end fashion. Is it therefore possible that some audio properties have different impact on their results? Overall performance being equal, answering this question would help to understand if one approach is preferable over the other under specific audio conditions. Among other possible factors (e.g. noise, recording conditions, overlapping speakers) we tried to shed light on this aspect by focusing on two common factors: audio duration and speech rate. To this aim, we grouped the sentences in the PE-set according to the sentence-wise HTER percentage difference  i.e. the difference between the cascade and direct HTER scores divided by their average. The threshold for considering performance differences as significant was set to 10%. The resulting groups contain sentences where: i) cascade is significantly better than direct, ii) direct is significantly better than cascade, iii) the difference between the two is not significant, and iv) both systems have HTER=0. For each group, we calculated the average audio duration and the corresponding speech rate in terms of phonemes 9 per second. Results are shown in Table 2, where  for the sake of completeness  also the length of the reference audio transcript is given, together with the average HTER of the systems. As we can see, results are coherent across languages: audio duration and speech rate averages do not differ, neither when one system performs significantly better than the other, nor when the HTER differences are not significant. We can hence conclude that, if audio duration and speech rate have any influence on systems' performance, our analysis does not highlight specific conditions that are more favorable to one approach than to the other. Both are equally robust with respect to the audio properties here considered. Handling the input audio differently, the two approaches have inherent strengths and weaknesses. In particular, although suffering from the wellknown scarcity of sizeable training corpora, direct solutions come with the promise (Sperber and Paulik, 2020) of: i) higher robustness to error propagation, and ii) reduced loss of speech information (e.g. prosody). Our next qualitative analysis tries to delve into these aspects by looking at audio understanding and prosody issues. Audio understanding. Errors due to wrong audio understanding are easy to identify for cascade systems  since they are evident in the intermediate ASR transcripts  but harder to spot for direct systems, whose internal representations are by far less accessible. In this case, errors can still be identified in mistranslations corresponding to words which are phonetically similar to parts of the input audio  e.g. nice voice mistranslated in German as nette Jungen ( nice boys ). To spot such errors, our annotators carefully inspected the PE-set by comparing the audio, the reference transcripts and systems' output translations for both the cascade and direct models, as well as the ASR transcripts for the cascade one. Some interesting examples of the identified errors are reported in Table 3. As shown in Table 4, audio understanding errors are quite common for both systems in all language pairs. However, both the number of errors and the number of sentences they affect is significantly lower for the direct one. We observed that this is the case especially for more difficult sentences, such as sentences with poor audio quality and overlapping or disfluent speech. Though far from being conclusive (we acknowledge that, due to the opacity of direct models, their error counts might be slightly underestimated), this analysis seems to confirm the theoretical advantages of direct ST. This finding advocates for more thorough future investigations on neural networks' interpretability, targeting its empirical verification on larger and diverse benchmarks. Prosody. Prosody is central to disambiguating utterances, as it reflects language elements which may not be encoded by grammar and vocabulary choices. While prosody is directly encoded by the direct system, it is lost in the unpunctuated input received by the MT component of a cascade. Besides few interrogative sentences, our annotators were able to isolate only a handful of utterances whose prosodic markers result in different interpretations by the two models. Concerning interrogatives, both systems managed to translate them correctly in most cases (24 for cascade and 25 for direct out of 31). This is not surprising given the syntactic structure of English questions, which is explicit and does not rely solely on prosody (e.g. compared to Italian). In all other cases (examples in Table 5), the direct model's higher sensitivity to prosody seems to give it an edge on cascade in disambiguating and correctly rendering the utterance meaning. Also this finding calls for future inquiries aimed to check the regularity of these differences on larger datasets. For this analysis, we rely on the publicly available tool 10 used by Bentivogli et al. (2018a) to analyse src nation states  governments doing the attacks C Regierungen der Nationalstaaten [governments of nation states] D Nationen, Regierungen [nations, governments] src like the one we saw before, moving C como el que vimos antes de moverse [like the one we saw before moving] D como el que hemos visto antes, moviendose [like the one we saw before, moving] src Photos like this: construction going on C Foto come questa costruzione [Photos like this construction] D Foto come queste: costruzione [Photos like these: construction] en-de en-es en-it C D % C D % C D % L 2481 2560 +3.2 2674 2497 -6.6 2264 2264 0.0 M 468 536 +14.5 535 494 -7.7 433 470 +8.6 R 398 476 +19.6 308 290 -5.8 230 226 -1.7 3347 3572 +6.7 3517 3281 -6.7 2927 2960 +1.1 Table 6: Distribution of (L)exical, (M)orphological and (R)eordering errors. what linguistic phenomena are best modeled by MT systems. The tool exploits manual post-edits and HTER-based computations to detect and classify translation errors according to three linguistic categories: lexicon, morphology and word order. Table 6 presents their distribution. As expected from the HTER scores in Table 1, results vary across language pairs. On en-it, systems show pretty much the same number of errors, with a slight percentage gain (+1.1) in favor of the cascade. For the other two pairs, differences are more marked and opposite, with an overall error reduction for the direct system on en-es (-6.7) and in favor of the cascade on en-de (+6.7). Looking at the distribution of errors across categories, while for en-es the direct system is always better and the percentage reduction is homogeneously distributed, for en-de the better performance of the cascade is concentrated in the morphology and word order categories. Since English and German are the most different languages in terms of morphology and word order, this result suggests that cascade systems still have an edge on the direct ones in their ability to handle morphology and word reordering. This is further supported by en-it: the only difference, in favor of the cascade, is indeed observed in the morphology category. Since lexical errors represent by far the most frequent category for both approaches in all language pairs, we complement the automatic analysis with a more fine-grained manual inspection, further distinguishing among lexical errors due to missing words, extra words, or wrong lexical choice. 11 The analysis was carried out on subsets of the PE-set, created in such a way to be suitable for manual annotation. Namely, we removed sentences for which the output of the two systems is: i) identical, ii) judged correct by post-editors (HTER=0), or iii) too poor to be reliably annotated for errors (HTER > 40%). The resulting sets contain 207 sentences for en-de, 238 for en-es, and 285 for en-it. This analysis reveals that, for all language pairs, wrong lexical choice is the most frequent error type (  65% of lexical errors on average) followed by missing words (  30%), and extra words (  5%). While errors due to lexical choice and superflu-ous words vary across languages, we observe a systematic behavior with respect to missing words (words that are present in the audio but are not translated). As we can see in Table 7, direct systems lose more information from the source input than their cascade counterparts, in terms of both single words and contiguous word sequences. It is particularly interesting to notice that also for en-es  where the direct system is significantly stronger than the cascade  the issue is still evident, although to a lesser extent. Table 8 collects examples of the encountered lexical phenomena. Finally, we report that a non-negligible amount of missing words (between 10% and 20%) is represented by discourse markers, i.e. words or phrases used to connect and manage what is being said (e.g. you know, well, now). Although this is 11 Various error taxonomies covering different levels of granularity have been developed, and the distinction between these types of lexical errors is widely adopted, including the DQF-MQM framework  https://info.taus.net/ dqf-mqm-error-typology-templ AUDIO That's fine, says George, C Das ist in Ordnung. [  ] George, D Das ist in Ordnung, [  ] George, AUDIO Well after two years, ... a frequent phenomenon in speech, not translating discourse markers cannot be properly considered as an error, since markers i) do not carry semantic information, and ii) can be intentionally dropped in some use cases, such as in subtitling. So far, our inquiry has been entirely driven by pre-defined assumptions (the importance of certain audio properties) and linguistic criteria (the focus on specific error types). This top-down approach, however, might fail to disclose important differences, which were not specifically sought after when analysing the two paradigms. This consideration motivates the adoption of the complementary bottom-up approach that concludes our comparative study by answering the question: is the output of cascade and direct systems distinguishable? Understanding if and why discriminating between the two is possible would not only suggest new issues to look at. It would also highlight possible output regularities that, despite the similar overall performance, make one paradigm preferable over the other in specific application scenarios. To this aim, we set up a classification experiment, comparing the ability of humans to correctly identify the output of the two systems with the performance of an automatic text classifier. After getting acquainted with systems' output through the previous manual analyses, our assessors were instructed to perform a classification task. The classification had to be performed on 10 blocks of items comprising a set of unseen English contiguous sentences (gold transcripts) from the MuST-C Common test set, and two sets of anonymized translations, one produced by the cascade and one by the direct model. For each block, the assessors had to assign each set of translations to the correct system, or label them as indistinguishable. To investigate whether more context helps in the assignment, we set up two experiments with respectively 10 and 20 contiguous sentences per block. The results in Table 9 show that en-es and en-it systems are not distinguishable, since only a maximum of 4 blocks out of 10 were correctly classified, while most en-de blocks were correctly classified. According to the en-de assessor, this is due to the fact that the structure of the sentences generated by the direct system is very similar to that of the corresponding English sources. This characteristic stands out in German, which differs from English in terms of word order more than Italian and Spanish. This type of behavior does not necessarily imply the presence of errors but, like a fingerprint, makes the en-de direct system more recognizable by a human. Furthermore, being sub-optimal for German, this structure can cause preferential edits by the post-editors, which would be in line with the concentration of errors in the word order category observed in Table 6 (+19.6%). Assessing the importance of context, the ability of humans to distinguish the systems does not improve when passing from 10 to 20 sentences per block. This suggests that the behavioral differences between cascade and direct systems are so subtle that, on larger samples, they mix up and balance making their fingerprints less traceable. As a complement to the human classification experiment, we check whether an automatic tool is able to accomplish a similar task. Our classifier combines n -gram language models with the Naive Bayes algorithm, as proposed in (Peng and Schu-urmans, 2003). We trained two 5-gram models, respectively using translations by the cascade and the direct systems. At classification time, given a translated text, the classifier computes the perplexity of the two models and assigns the cascade or direct label based on the model with the lowest perplexity. Also these experiments were carried out on the MuST-C Common set. The classifier was tested via k-fold cross-validation, for different values of k  i.e. different sizes of text to classify. As shown in Figure 1, contrary to humans, the more data the classifier receives, the higher its accuracy in discriminating between systems. Already at a size of 20 sentences, accuracy is always  80%. This suggests that systems have their own lan-guage, a fluency-related fingerprint. To check this finding, we measured outputs' lexical diversity in terms of moving average Type-Token Ratio  maTTR (Covington and McFall, 2010)  and with the Measure of Textual Lexical Diversity (MTLD) by McCarthy and Jarvis (2010). Table 10 shows that the cascade output exhibits higher lexical diversity on all languages, with smaller differences on en-de and en-es compared to en-it. A plausible conclusion is that the cascade produces richer output, whose variety does not necessarily result in better translations nor is appreciated by humans. Indeed, annotators were able to correctly distinguish the output only for en-de, where lexical diversity is similar (see  7.1). There is a time when the possible transition from consolidated technological frameworks to new emerging paradigms depends on answering fundamental questions about their potential, strengths and weaknesses. A time when technology developers are faced with the choice of where to direct their future investments. Five years after its appearance on the scene, the direct approach to ST confronts the community with similar questions in relation to the traditional cascade paradigm that it aims to overtake. Our investigation showed that, in spite of the known data paucity conditions still penalizing the direct approach, the two technologies now perform substantially on par. Subtle differences in their behavior exist: overall performance being equal, the cascade still seems to have an edge in terms of morphology, word ordering and lexical diversity, which is balanced by the advantages of direct models in audio understanding and in capturing prosody. However, they do not seem sufficient and consistent enough across languages to make the output of the two approaches easily distinguishable, nor to make one model preferable to the other. Back to our title, they no longer make a difference. We are aware that the generalizability of these results depends on several factors such as the considered languages, systems and benchmarks, as well as the human workforce deployed for the inquiry. Here, with the help of professionals, we proposed multi-faceted quantitative and qualitative analyses, run on the output of state-of-the-art systems on three language pairs  though, by now, covering only the most-explored and data-favorable condition, which has English as source. Although our findings hold for a specific scenario, in which free data were at our disposal (and to which we contribute back by releasing high-quality post-edits), they might not be generalizable to other (e.g. dif-ficult, distant) languages and other (e.g. highly specialized) domains. Nevertheless, we present them as a timely contribution towards answering a burning question within the ST community. The creation of the post-edits used in this work was funded by the European Association for Machine Translation (EAMT) through its 2020 Sponsorship of Activities programme. The computational costs were covered by the End-to-end Spoken Language Translation in Rich Data Conditions project, 12 which was financially supported by an Amazon AWS ML Grant.
abstain abstain method method objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain abstain objective objective other other other other other other other other other other other other other other other other other other other other other other other other other other other other other method abstain abstain result abstain abstain abstain result abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method other abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain result abstain abstain abstain result objective objective objective other other