Unsupervised Meta-Learning via In-Context Learning
Anna Vettoruzzo Lorenzo Braccaioli Joaquin Vanschoren Marlena Nowaczyk
Unsupervised meta-learning aims to learn feature representations from unsupervised datasets that can transfer to downstream tasks with limited labeled data.
In this paper, we propose a novel approach to unsupervised meta-learning that leverages the generalization abilities of in-context learning observed in transformer architectures. Our method reframes meta-learning as a sequence modeling problem, enabling the transformer encoder to learn task context from support images and utilize it to predict query images. 
At the core of our approach lies the creation of diverse tasks generated using a combination of data augmentations and a mixing strategy that challenges the model during training while fostering generalization to unseen tasks at test time. 
Experimental results on benchmark datasets showcase the superiority of our approach over existing unsupervised meta-learning baselines, establishing it as the new state-of-the-art. Remarkably, our method achieves competitive results with supervised and self-supervised approaches, underscoring its efficacy in leveraging generalization over memorization.
meta-learning, unsupervised learning, in-context learning
We introduce a novel unsupervised meta-learning approach that reframes meta-learning as a sequence modeling problem and leverages the generalization abilities of in-context learners.