title: A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning
summary: Pre-trained vision models (PVMs) are fundamental to modern robotics, yet
  their optimal configuration remains unclear. Through systematic evaluation, we
  find that while DINO and iBOT outperform MAE across visuomotor control and
  perception tasks, they struggle when trained on non-(single-)object-centric
  (NOC) data--a limitation strongly correlated with their diminished ability to
  learn object-centric representations. This investigation indicates that the
  ability to form object-centric representations from the non-object-centric
  robotics dataset is the key to success for PVMs. Motivated by this discovery,
  we designed SlotMIM, a method that induces object-centric representations by
  introducing a semantic bottleneck to reduce the number of prototypes to
  encourage the emergence of objectness as well as cross-view consistency
  regularization for encouraging multiview invariance. Our experiments encompass
  pre-training on object-centric, scene-centric, web-crawled, and ego-centric
  data. Across all settings, our approach learns transferrable representations
  and achieves significant improvements over prior work in image recognition,
  scene understanding, and robot learning evaluations. When scaled up with
  million-scale datasets, our method also demonstrates superior data efficiency
  and scalability. Our code and models are publicly available at
  https://github.com/CVMI-Lab/SlotMIM.
paper_authors._id: 67d0434c9507f2114701cf96
paper_authors.name: Xin Wen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:42:20.798Z
paper_authors._id: 63483629ac5172169929da0e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665676793089-noauth.jpeg
paper_authors.fullname: Xin Wen
paper_authors.type: user
paper_authors.user: xwen99
paper_authors._id: 67d0434c9507f2114701cf97
paper_authors.name: Bingchen Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:20:55.624Z
paper_authors._id: 62dcd71075e9787ec5aa41ba
paper_authors.avatarUrl: /avatars/f37ce036b76180ed0fa004f9c8c09363.svg
paper_authors.fullname: Bingchen Zhao
paper_authors.type: user
paper_authors.user: tennant
paper_authors._id: 67d0434c9507f2114701cf98
paper_authors.name: Yilun Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:23:40.661Z
paper_authors._id: 6647a9eb658c482f961680dc
paper_authors.avatarUrl: /avatars/7cdd80f7e78224efc4426c3911ccfc52.svg
paper_authors.fullname: Yilun Chen
paper_authors.type: user
paper_authors.user: allenchenyilun
paper_authors._id: 67d0434c9507f2114701cf99
paper_authors.name: Jiangmiao Pang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:20:26.954Z
paper_authors._id: 65783ee6ee33d547aecc3ffc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
paper_authors.fullname: Jiangmiao Pang
paper_authors.type: user
paper_authors.user: Jiangmiao
paper_authors._id: 67d0434c9507f2114701cf9a
paper_authors.name: Xiaojuan Qi
paper_title: A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning
paper_summary: Pre-trained vision models (PVMs) are fundamental to modern robotics, yet
  their optimal configuration remains unclear. Through systematic evaluation, we
  find that while DINO and iBOT outperform MAE across visuomotor control and
  perception tasks, they struggle when trained on non-(single-)object-centric
  (NOC) data--a limitation strongly correlated with their diminished ability to
  learn object-centric representations. This investigation indicates that the
  ability to form object-centric representations from the non-object-centric
  robotics dataset is the key to success for PVMs. Motivated by this discovery,
  we designed SlotMIM, a method that induces object-centric representations by
  introducing a semantic bottleneck to reduce the number of prototypes to
  encourage the emergence of objectness as well as cross-view consistency
  regularization for encouraging multiview invariance. Our experiments encompass
  pre-training on object-centric, scene-centric, web-crawled, and ego-centric
  data. Across all settings, our approach learns transferrable representations
  and achieves significant improvements over prior work in image recognition,
  scene understanding, and robot learning evaluations. When scaled up with
  million-scale datasets, our method also demonstrates superior data efficiency
  and scalability. Our code and models are publicly available at
  https://github.com/CVMI-Lab/SlotMIM.
paper_ai_keywords: DINO iBOT MAE object-centric representations non-(single-)object-centric (NOC) data SlotMIM semantic bottleneck prototypes objectness cross-view consistency regularization multiview invariance pre-training scene-centric data web-crawled data ego-centric data transferrable representations image recognition scene understanding robot learning data efficiency scalability