A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning
Pre-trained vision models (PVMs) are fundamental to modern robotics, yet
their optimal configuration remains unclear. Through systematic evaluation, we
find that while DINO and iBOT outperform MAE across visuomotor control and
perception tasks, they struggle when trained on non-(single-)object-centric
(NOC) data--a limitation strongly correlated with their diminished ability to
learn object-centric representations. This investigation indicates that the
ability to form object-centric representations from the non-object-centric
robotics dataset is the key to success for PVMs. Motivated by this discovery,
we designed SlotMIM, a method that induces object-centric representations by
introducing a semantic bottleneck to reduce the number of prototypes to
encourage the emergence of objectness as well as cross-view consistency
regularization for encouraging multiview invariance. Our experiments encompass
pre-training on object-centric, scene-centric, web-crawled, and ego-centric
data. Across all settings, our approach learns transferrable representations
and achieves significant improvements over prior work in image recognition,
scene understanding, and robot learning evaluations. When scaled up with
million-scale datasets, our method also demonstrates superior data efficiency
and scalability. Our code and models are publicly available at
https://github.com/CVMI-Lab/SlotMIM.
67d0434c9507f2114701cf96
Xin Wen
claimed_verified
2025-03-12T08:42:20.798Z
63483629ac5172169929da0e
https://cdn-avatars.huggingface.co/v1/production/uploads/1665676793089-noauth.jpeg
Xin Wen
user
xwen99
67d0434c9507f2114701cf97
Bingchen Zhao
admin_assigned
2025-03-11T14:20:55.624Z
62dcd71075e9787ec5aa41ba
/avatars/f37ce036b76180ed0fa004f9c8c09363.svg
Bingchen Zhao
user
tennant
67d0434c9507f2114701cf98
Yilun Chen
admin_assigned
2025-03-11T14:23:40.661Z
6647a9eb658c482f961680dc
/avatars/7cdd80f7e78224efc4426c3911ccfc52.svg
Yilun Chen
user
allenchenyilun
67d0434c9507f2114701cf99
Jiangmiao Pang
admin_assigned
2025-03-11T14:20:26.954Z
65783ee6ee33d547aecc3ffc
https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
Jiangmiao Pang
user
Jiangmiao
67d0434c9507f2114701cf9a
Xiaojuan Qi
claimed_verified
2025-07-15T19:13:40.396Z
6875266f9cd3191dfddc7071
/avatars/64c581910833b111e9a7bae5b8740229.svg
xiaojuan qi
user
xjqi
A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning
Pre-trained vision models (PVMs) are fundamental to modern robotics, yet
their optimal configuration remains unclear. Through systematic evaluation, we
find that while DINO and iBOT outperform MAE across visuomotor control and
perception tasks, they struggle when trained on non-(single-)object-centric
(NOC) data--a limitation strongly correlated with their diminished ability to
learn object-centric representations. This investigation indicates that the
ability to form object-centric representations from the non-object-centric
robotics dataset is the key to success for PVMs. Motivated by this discovery,
we designed SlotMIM, a method that induces object-centric representations by
introducing a semantic bottleneck to reduce the number of prototypes to
encourage the emergence of objectness as well as cross-view consistency
regularization for encouraging multiview invariance. Our experiments encompass
pre-training on object-centric, scene-centric, web-crawled, and ego-centric
data. Across all settings, our approach learns transferrable representations
and achieves significant improvements over prior work in image recognition,
scene understanding, and robot learning evaluations. When scaled up with
million-scale datasets, our method also demonstrates superior data efficiency
and scalability. Our code and models are publicly available at
https://github.com/CVMI-Lab/SlotMIM.
SlotMIM, a method introducing a semantic bottleneck and cross-view consistency regularization, enhances pre-trained vision models' ability to learn object-centric representations, leading to superior performance in various robotic tasks.
DINO iBOT MAE SlotMIM object-centric representations semantic bottleneck cross-view consistency regularization pre-training image recognition scene understanding robot learning data efficiency scalability