Less is More: Accurate Speech Recognition & Translation without Web-Scale Data
<|reference_start|>Less is More: Accurate Speech Recognition & Translation without Web-Scale Data: Recent advances in speech recognition and translation rely on hundreds of thousands of hours of Internet speech data. We argue that state-of-the art accuracy can be reached without relying on web-scale data. Canary - multilingual ASR and speech translation model, outperforms current state-of-the-art models - Whisper, OWSM, and Seamless-M4T on English, French, Spanish, and German languages, while being trained on an order of magnitude less data than these models. Three key factors enables such data-efficient model: (1) a FastConformer-based attention encoder-decoder architecture (2) training on synthetic data generated with machine translation and (3) advanced training techniques: data-balancing, dynamic data blending, dynamic bucketing and noise-robust fine-tuning. The model, weights, and training code will be open-sourced.<|reference_end|>
@article{puvvada2024less,
    title={Less is More: Accurate Speech Recognition & Translation without
  Web-Scale Data},
    author={Krishna C. Puvvada, Piotr .Zelasko, He Huang, Oleksii Hrinchuk,
  Nithin Rao Koluguri, Kunal Dhawan, Somshubra Majumdar, Elena Rastorgueva,
  Zhehuai Chen, Vitaly Lavrukhin, Jagadeesh Balam, Boris Ginsburg},
    journal={arXiv preprint arXiv:2406.19674},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.19674},
    primaryClass={cs.CL cs.LG cs.SD eess.AS}
}