All increases in the dataset size result in improved perfor-
mance on all tasks, although we see signiﬁcant variability
in improvement rates across tasks and sizes. Performance
improves rapidly on English speech recognition from 3,000
to 13,000 hours and then slows down noticeably between
13,000 and 54,000 hours. Using the full dataset, which cor-
responds to another 12.5× increase in size results in only a
further 1 point drop in WER. This mirrors the diminishing
returns observed with model size scaling for English speech
recognition and could similarly be explained by saturation
effects when approaching human-level performance.
Improvements in WER follow a power-law trend for mul-
tilingual speech recognition till 54,000 hours and then de-
viate from this trend, improving only a further 7 points
when increasing to the full dataset size. For X→en transla-
tion, performance is practically zero when training on 7,000
hours of audio or less, and then follows a roughly log-linear
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision