Soft Language Prompts for Language Transfer
<|reference_start|>Soft Language Prompts for Language Transfer: Cross-lingual knowledge transfer, especially between high- and low-resource languages, remains a challenge in natural language processing (NLP). This study offers insights for improving cross-lingual NLP applications through the combination of parameter-efficient fine-tuning methods. We systematically explore strategies for enhancing this cross-lingual transfer through the incorporation of language-specific and task-specific adapters and soft prompts. We present a detailed investigation of various combinations of these methods, exploring their efficiency across six languages, focusing on three low-resource languages, including the to our knowledge first use of soft language prompts. Our findings demonstrate that in contrast to claims of previous work, a combination of language and task adapters does not always work best; instead, combining a soft language prompt with a task adapter outperforms other configurations in many cases.<|reference_end|>
@article{vykopal2024soft,
    title={Soft Language Prompts for Language Transfer},
    author={Ivan Vykopal, Simon Ostermann, Mari'an v{S}imko},
    journal={arXiv preprint arXiv:2407.02317},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02317},
    primaryClass={cs.CL}
}