012345678WER on LibriSpeech dev-clean (%)01020304050Average WER on [Common Voice, CHiME-6, TED-LIUM] (%)Supervised LibriSpeech modelsZero-shot Whisper modelsZero-shot Human (Alec)Ideal robustness (y = x)Robust Speech Recognition via Large-Scale Weak Supervision

7

Figure 3. Correlation of pre-training supervision amount with
downstream speech recognition performance. The amount of
pre-training speech recognition data for a given language is very
predictive of zero-shot performance on that language in Fleurs.

Figure 4. Correlation of pre-training supervision amount with
downstream translation performance. The amount of pre-
training translation data for a given language is only moderately
predictive of Whisper’s zero-shot performance on that language in
Fleurs.

MLS VoxPopuli

Model
VP-10K + FT
XLS-R (1B)
mSLAM-CTC (2B)
Maestro
Zero-Shot Whisper

-

10.9
9.7
-
7.3

15.3
10.6
9.1
8.1
13.6
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision