No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement
<|reference_start|>No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement: Modular deep learning is the state-of-the-art solution for lifting the curse of multilinguality, preventing the impact of negative interference and enabling cross-lingual performance in Multilingual Pre-trained Language Models. However, a trade-off of this approach is the reduction in positive transfer learning from closely related languages. In response, we introduce a novel method called language arithmetic, which enables training-free post-processing to address this limitation. Extending the task arithmetic framework, we apply learning via addition to the language adapters, transitioning the framework from a multi-task to a multilingual setup. The effectiveness of the proposed solution is demonstrated on three downstream tasks in a MAD-X-based set of cross-lingual schemes, acting as a post-processing procedure. Language arithmetic consistently improves the baselines with significant gains, especially in the most challenging case of zero-shot application. Our code and models are available at https://github.com/mklimasz/language-arithmetic .<|reference_end|>
@article{klimaszewski2024no,
    title={No Train but Gain: Language Arithmetic for training-free Language
  Adapters enhancement},
    author={Mateusz Klimaszewski, Piotr Andruszkiewicz, Alexandra Birch},
    journal={arXiv preprint arXiv:2404.15737},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.15737},
    primaryClass={cs.CL}
}