Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers,
R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J.,
Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van
Kerkwijk, M. H., Brett, M., Haldane, A., Fern´andez del
R´ıo, J., Wiebe, M., Peterson, P., G´erard-Marchant, P.,
Sheppard, K., Reddy, T., Weckesser, W., Abbasi, H.,
Gohlke, C., and Oliphant, T. E. Array programming
with NumPy. Nature, 585:357–362, 2020. doi: 10.1038/
s41586-020-2649-2.

Robust Speech Recognition via Large-Scale Weak Supervision

16

Hendrycks, D. and Gimpel, K. Gaussian error linear units

(gelus). arXiv preprint arXiv:1606.08415, 2016.

Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan,
R., and Song, D. Pretrained transformers improve out-of-
distribution robustness. arXiv preprint arXiv:2004.06100,
2020.
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision