Improving Self-supervised Pre-training using Accent-Specific Codebooks
<|reference_start|>Improving Self-supervised Pre-training using Accent-Specific Codebooks: Speech accents present a serious challenge to the performance of state-of-the-art end-to-end Automatic Speech Recognition (ASR) systems. Even with self-supervised learning and pre-training of ASR models, accent invariance is seldom achieved. In this work, we propose an accent-aware adaptation technique for self-supervised learning that introduces a trainable set of accent-specific codebooks to the self-supervised architecture. These learnable codebooks enable the model to capture accent specific information during pre-training, that is further refined during ASR finetuning. On the Mozilla Common Voice dataset, our proposed approach outperforms all other accent-adaptation approaches on both seen and unseen English accents, with up to 9% relative reduction in word error rate (WER).<|reference_end|>
@article{prabhu2024improving,
    title={Improving Self-supervised Pre-training using Accent-Specific Codebooks},
    author={Darshan Prabhu, Abhishek Gupta, Omkar Nitsure, Preethi Jyothi, Sriram
  Ganapathy},
    journal={arXiv preprint arXiv:2407.03734},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.03734},
    primaryClass={cs.CL cs.AI cs.LG cs.SD eess.AS}
}