XB-MAML: Learning Expandable Basis Parameters for Effective Meta-Learning with Wide Task Coverage
<|reference_start|>XB-MAML: Learning Expandable Basis Parameters for Effective Meta-Learning with Wide Task Coverage: Meta-learning, which pursues an effective initialization model, has emerged as a promising approach to handling unseen tasks. However, a limitation remains to be evident when a meta-learner tries to encompass a wide range of task distribution, e.g., learning across distinctive datasets or domains. Recently, a group of works has attempted to employ multiple model initializations to cover widely-ranging tasks, but they are limited in adaptively expanding initializations. We introduce XB-MAML, which learns expandable basis parameters, where they are linearly combined to form an effective initialization to a given task. XB-MAML observes the discrepancy between the vector space spanned by the basis and fine-tuned parameters to decide whether to expand the basis. Our method surpasses the existing works in the multi-domain meta-learning benchmarks and opens up new chances of meta-learning for obtaining the diverse inductive bias that can be combined to stretch toward the effective initialization for diverse unseen tasks.<|reference_end|>
@article{lee2024xb-maml:,
    title={XB-MAML: Learning Expandable Basis Parameters for Effective
  Meta-Learning with Wide Task Coverage},
    author={Jae-Jun Lee, Sung Whan Yoon},
    journal={arXiv preprint arXiv:2403.06768},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.06768},
    primaryClass={cs.LG}
}