the hope that it will learn to use longer-range text context
to resolve ambiguous audio. Speciﬁcally, with some proba-
bility we add the transcript text preceding the current audio
segment to the decoder’s context. We indicate the beginning
of prediction with a <|startoftranscript|> token.
First, we predict the language being spoken which is repre-
sented by a unique token for each language in our training
set (99 total). These language targets are sourced from the
aforementioned VoxLingua107 model. In the case where
there is no speech in an audio segment, the model is trained
to predict a <|nospeech|> token indicating this. The
next token speciﬁes the task (either transcription or trans-
lation) with an <|transcribe|> or <|translate|>
token. After this, we specify whether to predict timestamps
or not by including a <|notimestamps|> token for that
case. At this point, the task and desired format is fully
speciﬁed, and the output begins. For timestamp predic-
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision