Mitigating the Linguistic Gap with Phonemic Representations for Robust Multilingual Language Understanding
<|reference_start|>Mitigating the Linguistic Gap with Phonemic Representations for Robust Multilingual Language Understanding: Approaches to improving multilingual language understanding often require multiple languages during the training phase, rely on complicated training techniques, and -- importantly -- struggle with significant performance gaps between high-resource and low-resource languages. We hypothesize that the performance gaps between languages are affected by linguistic gaps between those languages and provide a novel solution for robust multilingual language modeling by employing phonemic representations (specifically, using phonemes as input tokens to LMs rather than subwords). We present quantitative evidence from three cross-lingual tasks that demonstrate the effectiveness of phonemic representation, which is further justified by a theoretical analysis of the cross-lingual performance gap.<|reference_end|>
@article{jung2024mitigating,
    title={Mitigating the Linguistic Gap with Phonemic Representations for Robust
  Cross-lingual Transfer},
    author={Haeji Jung, Changdae Oh, Jooeon Kang, Jimin Sohn, Kyungwoo Song,
  Jinkyu Kim, David R. Mortensen},
    journal={arXiv preprint arXiv:2402.14279},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.14279},
    primaryClass={cs.CL cs.AI}
}