by ﬁne-tuning a prototype model trained on a prototype ver-
sion of the dataset on VoxLingua107 (Valk & Alum¨ae, 2021)
to ensure that the spoken language matches the language of
the transcript according to CLD2. If the two do not match,
we don’t include the (audio, transcript) pair as a speech
recognition training example in the dataset. We make an
exception if the transcript language is English and add these
pairs to the dataset as X→en speech translation training
examples instead. We use fuzzy de-duping of transcript
texts to reduce the amount of duplication and automatically
generated content in the training dataset.
We break audio ﬁles into 30-second segments paired with
the subset of the transcript that occurs within that time
segment. We train on all audio, including segments where
there is no speech (though with sub-sampled probability)
and use these segments as training data for voice activity
detection.
For an additional ﬁltering pass, after training an initial model
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision