Robust Speech Recognition via Large-Scale Weak Supervision

2
2
0
2

 
c
e
D
6

 

 
 
]
S
A
.
s
s
e
e
[
 
 

1
v
6
5
3
4
0

.

2
1
2
2
:
v
i
X
r
a

Alec Radford * 1 Jong Wook Kim * 1 Tao Xu 1 Greg Brockman 1 Christine McLeavey 1 Ilya Sutskever 1

Abstract

We study the capabilities of speech processing
systems trained simply to predict large amounts of
transcripts of audio on the internet. When scaled
to 680,000 hours of multilingual and multitask
supervision, the resulting models generalize well
to standard benchmarks and are often competitive
with prior fully supervised results but in a zero-
shot transfer setting without the need for any ﬁne-
tuning. When compared to humans, the models
approach their accuracy and robustness. We are
releasing models and inference code to serve as
a foundation for further work on robust speech
processing.
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision