AfriHuBERT: A self-supervised speech representation model for African languages
<|reference_start|>AfriHuBERT: A self-supervised speech representation model for African languages: In this work, we present AfriHuBERT, an extension of mHuBERT-147, a state-of-the-art (SOTA) and compact self-supervised learning (SSL) model, originally pretrained on 147 languages. While mHuBERT-147 was pretrained on 16 African languages, we expand this to cover 39 African languages through continued pretraining on 6,500+ hours of speech data aggregated from diverse sources, including 23 newly added languages. We evaluate AfriHuBERT on two key speech tasks: Language Identification (LID) and Automatic Speech Recognition (ASR) using FLEURS dataset. Our results show a +4% F1 score improvement on average for LID and a -1.2% average Word Error Rate (WER) reduction for ASR. Further analysis shows that ASR models trained on AfriHuBERT exhibit improved cross-corpus generalization. Additionally, the analysis indicates that the FLEURS have data quality limitations that may affect their suitability for evaluating low-resource African languages, suggesting the need for better evaluation benchmarks for these languages.<|reference_end|>
@article{alabi2024afrihubert:,
    title={AfriHuBERT: A self-supervised speech representation model for African
  languages},
    author={Jesujoba O. Alabi, Xuechen Liu, Dietrich Klakow, Junichi Yamagishi},
    journal={arXiv preprint arXiv:2409.20201},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.20201},
    primaryClass={cs.CL cs.SD eess.AS}
}