Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM
<|reference_start|>Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM: Multilingual large language models (LLMs) are great translators, but this is largely limited to high-resource languages. For many LLMs, translating in and out of low-resource languages remains a challenging task. To maximize data efficiency in this low-resource setting, we introduce Mufu, which includes a selection of automatically generated multilingual candidates and an instruction to correct inaccurate translations in the prompt. Mufu prompts turn a translation task into a postediting one, and seek to harness the LLM's reasoning capability with auxiliary translation candidates, from which the model is required to assess the input quality, align the semantics cross-lingually, copy from relevant inputs and override instances that are incorrect. Our experiments on En-XX translations over the Flores-200 dataset show LLMs finetuned against Mufu-style prompts are robust to poor quality auxiliary translation candidates, achieving performance superior to NLLB 1.3B distilled model in 64% of low- and very-low-resource language pairs. We then distill these models to reduce inference cost, while maintaining on average 3.1 chrF improvement over finetune-only baseline in low-resource translations.<|reference_end|>
@article{lim2024mufu:,
    title={Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM},
    author={Zheng Wei Lim, Nitish Gupta, Honglin Yu, Trevor Cohn},
    journal={arXiv preprint arXiv:2409.13949},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.13949},
    primaryClass={cs.CL}
}