Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models
<|reference_start|>Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models: Parameter-efficient fine-tuning (PEFT) methods are increasingly used with pre-trained language models (PLMs) for continual learning (CL). These methods involve training a PEFT module for each new task and using similarity-based selection to route modules during inference. However, they face two major limitations: 1) interference with already learned modules and 2) suboptimal routing when composing modules. In this paper, we introduce a method that isolates the training of PEFT modules for task specialization. Then, before evaluation, it learns to compose the previously learned modules by training a router that leverages samples from a small memory. We evaluate our method in two CL setups using several benchmarks. Our results show that our method provides a better composition of PEFT modules, leading to better generalization and performance compared to previous methods.<|reference_end|>
@article{araujo2024learning,
    title={Learning to Route for Dynamic Adapter Composition in Continual Learning
  with Language Models},
    author={Vladimir Araujo, Marie-Francine Moens, Tinne Tuytelaars},
    journal={arXiv preprint arXiv:2408.09053},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.09053},
    primaryClass={cs.LG cs.AI cs.CL}
}