Likhomanenko, T., Xu, Q., Pratap, V., Tomasello, P., Kahn,
J., Avidov, G., Collobert, R., and Synnaeve, G. Rethink-
ing evaluation in asr: Are our models robust enough?
arXiv preprint arXiv:2010.11745, 2020.

Loshchilov, I. and Hutter, F. Decoupled weight decay regu-

larization. arXiv preprint arXiv:1711.05101, 2017.

Luong, M.-T., Le, Q. V., Sutskever, I., Vinyals, O., and
Kaiser, L. Multi-task sequence to sequence learning.
arXiv preprint arXiv:1511.06114, 2015.

Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri,
M., Li, Y., Bharambe, A., and Van Der Maaten, L. Ex-
ploring the limits of weakly supervised pretraining. In
Proceedings of the European conference on computer
vision (ECCV), pp. 181–196, 2018.

Mauch, M. and Ewert, S. The audio degradation toolbox and
its application to robustness evaluation. In Proceedings of
the 14th International Society for Music Information Re-
trieval Conference (ISMIR 2013), Curitiba, Brazil, 2013.
accepted.
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision