Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition
<|reference_start|>Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition: This paper addresses the challenge of integrating low-resource languages into multilingual automatic speech recognition (ASR) systems. We introduce a novel application of weighted cross-entropy, typically used for unbalanced datasets, to facilitate the integration of low-resource languages into pre-trained multilingual ASR models within the context of continual multilingual learning. We fine-tune the Whisper multilingual ASR model on five high-resource languages and one low-resource language, employing language-weighted dynamic cross-entropy and data augmentation. The results show a remarkable 6.69% word error rate (WER) reduction for the low-resource language compared to the fine-tuned model without applying our approach, and a 48.86% WER reduction compared to the original Whisper model. In addition, our approach yields an average WER reduction of 3.29% across the six languages, showing no degradation for the high-resource languages.<|reference_end|>
@article{piñeiro-martín2024weighted,
    title={Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech
  Recognition},
    author={Andr'es Pi~neiro-Mart'in, Carmen Garc'ia-Mateo, Laura
  Doc'io-Fern'andez, Mar'ia del Carmen L'opez-P'erez, Georg Rehm},
    journal={Proceedings of Interspeech 2024},
    year={2024},
    doi={10.21437/Interspeech.2024-734},
    archivePrefix={arXiv},
    eprint={2409.16954},
    primaryClass={cs.CL cs.SD eess.AS}
}