5.3
5.4
6.0
6.5
7.4
10.5

12.8
11.3
11.6
12.0
12.6
16.0

10 CONCLUSION

We introduce Distil-Whisper, a distilled version of Whisper that is 49% smaller, 5.8 times faster,
and within 1% WER performance on OOD short-form audio. On OOD long-form audio, Distil-
Whisper outperforms Whisper, due to fewer hallucinations and repetitions. We show that large-scale
pseudo-labelling is an effective strategy for distilling ASR models, in particular when combined our
WER threshold filter. We further demonstrate that Distil-Whisper can be used in combination with
Whisper using speculative decoding to obtain the same outputs as the original model with 2 times
faster inference.

11 ACKNOWLEGEMENTS
DISTIL-WHISPER