Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen,
Bo Li, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Andrew Rosenberg, Rohit Prab-
havalkar, Daniel S. Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, Trevor
Strohman, Bhuvana Ramabhadran, Tara Sainath, Pedro Moreno, Chung-Cheng Chiu, Johan
Schalkwyk, Franc¸oise Beaufays, and Yonghui Wu. Google USM: Scaling Automatic Speech
Recognition Beyond 100 Languages. arXiv e-prints, art. arXiv:2303.01037, March 2023. doi:
10.48550/arXiv.2303.01037.

Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh,
Sanjiv Kumar, Jean-Franc¸ois Kagy, and Rishabh Agarwal. DistillSpec: Improving Speculative
Decoding via Knowledge Distillation. arXiv e-prints, art. arXiv:2310.08461, October 2023. doi:
10.48550/arXiv.2310.08461.

A ADDITIONAL DATASET DETAILS

A.1 TRAINING DATA
DISTIL-WHISPER