PictSure: Pretraining Embeddings Matters for In-Context Learning Image
  Classifiers
Building image classification models remains cumbersome in data-scarce
domains, where collecting large labeled datasets is impractical. In-context
learning (ICL) has emerged as a promising paradigm for few-shot image
classification (FSIC), enabling models to generalize across domains without
gradient-based adaptation. However, prior work has largely overlooked a
critical component of ICL-based FSIC pipelines: the role of image embeddings.
In this work, we present PictSure, an ICL framework that places the embedding
model -- its architecture, pretraining, and training dynamics -- at the center
of analysis. We systematically examine the effects of different visual encoder
types, pretraining objectives, and fine-tuning strategies on downstream FSIC
performance. Our experiments show that the training success and the
out-of-domain performance are highly dependent on how the embedding models are
pretrained. Consequently, PictSure manages to outperform existing ICL-based
FSIC models on out-of-domain benchmarks that differ significantly from the
training distribution, while maintaining comparable results on in-domain tasks.
Code can be found at https://github.com/PictSure/pictsure-library.
6854007941586a7b0193a0f9
Lukas Schiesser
extracted_confirmed
2025-06-19T12:20:53.136Z
64ba4e0bb14fad24e03d4afc
/avatars/65e402f5082c44ba7df2c3ed627930ce.svg
Lukas Schießer
user
lschiesser
6854007941586a7b0193a0fa
Cornelius Wolff
extracted_confirmed
2025-06-20T12:23:08.609Z
65d65a40531e0bc924f0b1a3
https://cdn-avatars.huggingface.co/v1/production/uploads/65d65a40531e0bc924f0b1a3/TWS4x4mT2N8mz7ne2U11u.png
Cornelius Wolff
user
cwolff
6854007941586a7b0193a0fb
Sophie Haas
extracted_confirmed
2025-06-19T12:30:36.214Z
67851a64806abc109d1a3764
/avatars/0eb0056334f5b496a31b8085c26d3036.svg
Sophie Haas
user
sohaas
6854007941586a7b0193a0fc
Simon Pukrop
extracted_confirmed
2025-06-19T12:20:54.777Z
655b6a9dc69231073903567c
/avatars/8b46f501d5877e463c3c7a1b2cd08126.svg
Simon Pukrop
user
SimonPukrop
PictSure: Pretraining Embeddings Matters for In-Context Learning Image
  Classifiers
Building image classification models remains cumbersome in data-scarce
domains, where collecting large labeled datasets is impractical. In-context
learning (ICL) has emerged as a promising paradigm for few-shot image
classification (FSIC), enabling models to generalize across domains without
gradient-based adaptation. However, prior work has largely overlooked a
critical component of ICL-based FSIC pipelines: the role of image embeddings.
In this work, we present PictSure, an ICL framework that places the embedding
model -- its architecture, pretraining, and training dynamics -- at the center
of analysis. We systematically examine the effects of different visual encoder
types, pretraining objectives, and fine-tuning strategies on downstream FSIC
performance. Our experiments show that the training success and the
out-of-domain performance are highly dependent on how the embedding models are
pretrained. Consequently, PictSure manages to outperform existing ICL-based
FSIC models on out-of-domain benchmarks that differ significantly from the
training distribution, while maintaining comparable results on in-domain tasks.
Code can be found at https://github.com/PictSure/pictsure-library.
PictSure is an in-context learning framework that enhances few-shot image classification by optimizing embedding models' architecture, pretraining, and fine-tuning strategies to improve out-of-domain performance.
in-context learning few-shot image classification embedding models visual encoder types pretraining objectives fine-tuning strategies