decoding algorithmically ensures that predictions of the main model are unchanged, meaning it can
be used as a drop-in replacement for existing speech recognition pipelines using Whisper.
Our work suggests that large-scale pseudo-labelling of speech data has been under-explored and
that it provides a promising technique for KD. To serve as a basis for further research on distillation
for speech recognition, we release training code, inference code and models under https://
github.com/huggingface/distil-whisper.
DISTIL-WHISPER