Zero-Shot Object-Centric Representation Learning
<|reference_start|>Zero-Shot Object-Centric Representation Learning: The goal of object-centric representation learning is to decompose visual scenes into a structured representation that isolates the entities. Recent successes have shown that object-centric representation learning can be scaled to real-world scenes by utilizing pre-trained self-supervised features. However, so far, object-centric methods have mostly been applied in-distribution, with models trained and evaluated on the same dataset. This is in contrast to the wider trend in machine learning towards general-purpose models directly applicable to unseen data and tasks. Thus, in this work, we study current object-centric methods through the lens of zero-shot generalization by introducing a benchmark comprising eight different synthetic and real-world datasets. We analyze the factors influencing zero-shot performance and find that training on diverse real-world images improves transferability to unseen scenarios. Furthermore, inspired by the success of task-specific fine-tuning in foundation models, we introduce a novel fine-tuning strategy to adapt pre-trained vision encoders for the task of object discovery. We find that the proposed approach results in state-of-the-art performance for unsupervised object discovery, exhibiting strong zero-shot transfer to unseen datasets.<|reference_end|>
@article{didolkar2024zero-shot,
    title={Zero-Shot Object-Centric Representation Learning},
    author={Aniket Didolkar and Andrii Zadaianchuk and Anirudh Goyal and Mike
  Mozer and Yoshua Bengio and Georg Martius and Maximilian Seitzer},
    journal={arXiv preprint arXiv:2408.09162},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.09162},
    primaryClass={cs.CV cs.LG}
}