ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning
<|reference_start|>ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning: Meta-learning enables learning systems to adapt quickly to new tasks, similar to humans. To emulate this human-like rapid learning and enhance alignment and discrimination abilities, we propose ConML, a universal meta-learning framework that can be applied to various meta-learning algorithms without relying on specific model architectures nor target models. The core of ConML is task-level contrastive learning, which extends contrastive learning from the representation space in unsupervised learning to the model space in meta-learning. By leveraging task identity as an additional supervision signal during meta-training, we contrast the outputs of the meta-learner in the model space, minimizing inner-task distance (between models trained on different subsets of the same task) and maximizing inter-task distance (between models from different tasks). We demonstrate that ConML integrates seamlessly with optimization-based, metric-based, and amortization-based meta-learning algorithms, as well as in-context learning, resulting in performance improvements across diverse few-shot learning tasks.<|reference_end|>
@article{wu2024conml:,
    title={ConML: A Universal Meta-Learning Framework with Task-Level Contrastive
  Learning},
    author={Shiguang Wu, Yaqing Wang, Yatao Bian, Quanming Yao},
    journal={arXiv preprint arXiv:2410.05975},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.05975},
    primaryClass={cs.LG}
}