Robustness The question of how effectively models trans-
fer and how robust they are to distribution shift and other
types of perturbations has long been studied and is actively
being researched across many ﬁelds of machine learning.
Torralba & Efros (2011) highlighted the lack of generaliza-
tion of machine learning models between datasets over a
decade ago. Many other works have shown and continu-
ally reiterated how despite high performance on IID test
sets, machine learning models can still make many mistakes
when evaluated in even slightly different settings (Lake et al.,
2017; Jia & Liang, 2017; Alcorn et al., 2019; Barbu et al.,
2019; Recht et al., 2019). More recently, Taori et al. (2020)
studied the robustness of image classiﬁcation models, and
Miller et al. (2020) investigated this for question-answering
models. A key ﬁnding has been that multi-domain train-
ing increases robustness and generalization as discussed in
the Introduction. This ﬁnding has been replicated across
RobustSpeechRecognitionviaLarge-ScaleWeakSupervision