Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the Knowledge in a Neural Network. arXiv

e-prints, art. arXiv:1503.02531, March 2015. doi: 10.48550/arXiv.1503.02531.

Jeremy Howard and Sebastian Ruder. Universal Language Model Fine-tuning for Text Classifica-
tion. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics
(Volume 1: Long Papers), pp. 328–339, Melbourne, Australia, July 2018. Association for Com-
putational Linguistics. doi: 10.18653/v1/P18-1031. URL https://aclanthology.org/
P18-1031.

Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov,
and Abdelrahman Mohamed. HuBERT: Self-Supervised Speech Representation Learning by
Masked Prediction of Hidden Units. arXiv e-prints, art. arXiv:2106.07447, June 2021. doi:
10.48550/arXiv.2106.07447.
DISTIL-WHISPER