Are We Done with Object-Centric Learning?
Object-centric learning (OCL) seeks to learn representations that only encode
an object, isolated from other objects or background cues in a scene. This
approach underpins various aims, including out-of-distribution (OOD)
generalization, sample-efficient composition, and modeling of structured
environments. Most research has focused on developing unsupervised mechanisms
that separate objects into discrete slots in the representation space,
evaluated using unsupervised object discovery. However, with recent
sample-efficient segmentation models, we can separate objects in the pixel
space and encode them independently. This achieves remarkable zero-shot
performance on OOD object discovery benchmarks, is scalable to foundation
models, and can handle a variable number of slots out-of-the-box. Hence, the
goal of OCL methods to obtain object-centric representations has been largely
achieved. Despite this progress, a key question remains: How does the ability
to separate objects within a scene contribute to broader OCL objectives, such
as OOD generalization? We address this by investigating the OOD generalization
challenge caused by spurious background cues through the lens of OCL. We
propose a novel, training-free probe called Object-Centric
Classification with Applied Masks (OCCAM), demonstrating that
segmentation-based encoding of individual objects significantly outperforms
slot-based OCL methods. However, challenges in real-world applications remain.
We provide the toolbox for the OCL community to use scalable object-centric
representations, and focus on practical applications and fundamental questions,
such as understanding object perception in human cognition. Our code is
available https://github.com/AlexanderRubinstein/OCCAM{here}.
67f7826a8b50772851ccb603
Alexander Rubinstein
admin_assigned
2025-04-10T08:52:04.485Z
64198d7efdfc2970b350f48f
/avatars/c0a0f30e1cbc22f1eb6bbc4549a5709c.svg
Alexander Rubinstein
user
arubique
67f7826a8b50772851ccb604
Ameya Prabhu
admin_assigned
2025-04-10T08:52:11.053Z
6464a0d41683d3c81f51924a
https://cdn-avatars.huggingface.co/v1/production/uploads/6464a0d41683d3c81f51924a/s7yYVwfUB4WOhVFJS6A6T.jpeg
Ameya Prabhu
user
AmeyaPrabhu
67f7826a8b50772851ccb605
Matthias Bethge
67f7826a8b50772851ccb606
Seong Joon Oh
admin_assigned
2025-04-10T08:52:30.797Z
638a50450f10aa3064f03f23
/avatars/0c068458e42950c851758a238225c3a6.svg
Seong Joon Oh
user
coallaoh
Are We Done with Object-Centric Learning?
Object-centric learning (OCL) seeks to learn representations that only encode
an object, isolated from other objects or background cues in a scene. This
approach underpins various aims, including out-of-distribution (OOD)
generalization, sample-efficient composition, and modeling of structured
environments. Most research has focused on developing unsupervised mechanisms
that separate objects into discrete slots in the representation space,
evaluated using unsupervised object discovery. However, with recent
sample-efficient segmentation models, we can separate objects in the pixel
space and encode them independently. This achieves remarkable zero-shot
performance on OOD object discovery benchmarks, is scalable to foundation
models, and can handle a variable number of slots out-of-the-box. Hence, the
goal of OCL methods to obtain object-centric representations has been largely
achieved. Despite this progress, a key question remains: How does the ability
to separate objects within a scene contribute to broader OCL objectives, such
as OOD generalization? We address this by investigating the OOD generalization
challenge caused by spurious background cues through the lens of OCL. We
propose a novel, training-free probe called Object-Centric
Classification with Applied Masks (OCCAM), demonstrating that
segmentation-based encoding of individual objects significantly outperforms
slot-based OCL methods. However, challenges in real-world applications remain.
We provide the toolbox for the OCL community to use scalable object-centric
representations, and focus on practical applications and fundamental questions,
such as understanding object perception in human cognition. Our code is
available https://github.com/AlexanderRubinstein/OCCAM{here}.
Segmentation-based OCL methods achieve superior zero-shot OOD performance compared to slot-based methods and provide tools for practical applications.
object-centric learning (OCL) out-of-distribution (OOD) generalization unsupervised object discovery pixel space segmentation models slot-based OCL Object-Centric Classification with Applied Masks (OCCAM)