Teaching Large Language Models an Unseen Language on the Fly
<|reference_start|>Teaching Large Language Models an Unseen Language on the Fly: Existing large language models struggle to support numerous low-resource languages, particularly the extremely low-resource ones, for which there is minimal training data available for effective parameter updating. We thus investigate whether LLMs can learn a new language on the fly solely through prompting. To study this question, we collect a research suite for Zhuang, a language supported by no LLMs currently. We introduce DiPMT++, a framework for adapting LLMs to unseen languages by in-context learning. Using a dictionary and 5K parallel sentences only, DiPMT++ significantly enhances the performance of GPT-4 from 0 to 16 BLEU for Chinese-to-Zhuang translation and achieves 32 BLEU for Zhuang-to-Chinese translation. We also validate the effectiveness of our framework on Kalamang, another unseen language. Furthermore, we demonstrate the practical utility of DiPMT++ in aiding humans in translating completely unseen languages, which could contribute to the preservation of linguistic diversity.<|reference_end|>
@article{zhang2024teaching,
    title={Teaching Large Language Models an Unseen Language on the Fly},
    author={Chen Zhang, Xiao Liu, Jiuheng Lin, Yansong Feng},
    journal={arXiv preprint arXiv:2402.19167},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.19167},
    primaryClass={cs.CL}
}