The objective of Distil-Whisper is to compress the original Whisper model into a smaller, faster vari-
ant of the model that retains its robustness to different acoustic conditions (speakers, speaking styles
and domains). To investigate this capability, we employ a broad collection of speech recognition
datasets to examine whether Distil-Whisper can effectively generalise across datasets and domains.
We evaluate the Distil-Whisper model on a total of 15 short-form datasets. The first 11 of these
datasets are the corresponding test splits for the training data used to distil the model. These test
splits are in distribution (ID) with the training data. The remaining four datasets are used as test
sets only, where Distil-Whisper is assessed in a zero-shot setting, that is without the use of the
corresponding training data, thereby measuring the model’s ability to generalise to out of distribution
DISTIL-WHISPER