How is the speech alignment quality evaluated?
Through a 3-point scale by annotators.
We present a corpus of sentence-aligned triples of German audio, German text, and English translation, based on German audio books. The corpus consists of over 100 hours of audio material and over 50k parallel sentences. The audio data is read speech and thus low in disfluencies. The quality of audio and sentence alignments has been checked by a manual evaluation, showing that speech alignment quality is in general very high. The sentence alignment quality is comparable to well-used parallel translation data and can be adjusted by cutoffs on the automatic alignment score. To our knowledge, this corpus is to date the largest resource for end-to-end speech translation for German.
Direct speech translation has recently been shown to be feasible using a single sequence-to-sequence neural model, trained on parallel data consisting of source audio, source text and target text. The crucial advantage of such end-to-end approaches is the avoidance of error propagation as in a pipeline approaches of speech recognition and text translation. While cascaded approaches have an advantage in that they can straightforwardly use large independent datasets for speech recognition and text translation, clever sharing of sub-networks via multi-task learning and two-stage modeling BIBREF0, BIBREF1, BIBREF2 has closed the performance gap between end-to-end and pipeline approaches. However, end-to-end neural speech translation is very data hungry while available datatsets must be considered large if they exceed 100 hours of audio. For example, the widely used Fisher and Call-home Spanish-English corpus BIBREF3 comprises 162 hours of audio and $138,819$ parallel sentences. Larger corpora for end-to-end speech translation have only recently become available for speech translation from English sources. For example, 236 hours of audio and $131,395$ parallel sentences are available for English-French speech translation based on audio books BIBREF4, BIBREF5. For speech translation of English TED talks, 400-500 hours of audio aligned to around $250,000$ parallel sentences depending on the language pair have been provided for eight target languages by DiGangiETAL:19. Pure speech recognition data are available in amounts of $1,000$ hours of read English speech and their transcriptions in the LibriSpeech corpus provided by PanayotovETAL:15. When it comes to German sources, the situation regarding corpora for end-to-end speech translation as well as for speech recognition is dire. To our knowledge, the largest freely available corpora for German-English speech translation comprise triples for 37 hours of German audio, German transcription, and English translation BIBREF6. Pure speech recognition data are available from 36 hours BIBREF7 to around 200 hours BIBREF8. We present a corpus of sentence-aligned triples of German audio, German text, and English translation, based on German audio books. The corpus consists of over 100 hours of audio material aligned to over 50k parallel sentences. Our approach mirrors that of KocabiyikogluETAL:18 in that we start from freely available audio books. The fact that the audio data is read speech keeps the number of disfluencies low. Furthermore, we use state-of-the art tools for audio-text and text-text alignment, and show in a manual evaluation that the speech alignment quality is in general very high, while the sentence alignment quality is comparable to widely used corpora such as that of KocabiyikogluETAL:18 and can be adjusted by cutoffs on the automatic alignment score. To our knowledge, the presented corpus is to data the largest resource for end-to-end speech translation for German.