Self-Translate-Train: Enhancing Cross-Lingual Transfer of Large Language Models via Inherent Capability
<|reference_start|>Self-Translate-Train: Enhancing Cross-Lingual Transfer of Large Language Models via Inherent Capability: Zero-shot cross-lingual transfer by fine-tuning multilingual pretrained models shows promise for low-resource languages, but often suffers from misalignment of internal representations between languages. We hypothesize that even when the model cannot generalize across languages effectively in fine-tuning, it still captures cross-lingual correspondence useful for cross-lingual transfer. We explore this hypothesis with Self-Translate-Train, a method that lets large language models (LLMs) to translate training data into the target language and fine-tunes the model on its own generated data. By demonstrating that Self-Translate-Train outperforms zero-shot transfer, we encourage further exploration of better methods to elicit cross-lingual capabilities of LLMs.<|reference_end|>
@article{ri2024self-translate-train:,
    title={Self-Translate-Train: Enhancing Cross-Lingual Transfer of Large Language
  Models via Inherent Capability},
    author={Ryokan Ri, Shun Kiyono, Sho Takase},
    journal={arXiv preprint arXiv:2407.00454},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.00454},
    primaryClass={cs.CL}
}