title: DINeMo: Learning Neural Mesh Models with no 3D Annotations
summary: Category-level 3D/6D pose estimation is a crucial step towards comprehensive
  3D scene understanding, which would enable a broad range of applications in
  robotics and embodied AI. Recent works explored neural mesh models that
  approach a range of 2D and 3D tasks from an analysis-by-synthesis perspective.
  Despite the largely enhanced robustness to partial occlusion and domain shifts,
  these methods depended heavily on 3D annotations for part-contrastive learning,
  which confines them to a narrow set of categories and hinders efficient
  scaling. In this work, we present DINeMo, a novel neural mesh model that is
  trained with no 3D annotations by leveraging pseudo-correspondence obtained
  from large visual foundation models. We adopt a bidirectional
  pseudo-correspondence generation method, which produce pseudo correspondence
  utilize both local appearance features and global context information.
  Experimental results on car datasets demonstrate that our DINeMo outperforms
  previous zero- and few-shot 3D pose estimation by a wide margin, narrowing the
  gap with fully-supervised methods by 67.3%. Our DINeMo also scales effectively
  and efficiently when incorporating more unlabeled images during training, which
  demonstrate the advantages over supervised learning methods that rely on 3D
  annotations. Our project page is available at
  https://analysis-by-synthesis.github.io/DINeMo/.
paper_authors._id: 67e4b035af7d0551dc377e13
paper_authors.name: Weijie Guo
paper_authors._id: 67e4b035af7d0551dc377e14
paper_authors.name: Guofeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:15:46.953Z
paper_authors._id: 661434dd0dcdede49138cd7c
paper_authors.avatarUrl: /avatars/35f08bf08c5e9edfb3c78e280af718cb.svg
paper_authors.fullname: Guofeng Zhang
paper_authors.type: user
paper_authors.user: guofeng1123
paper_authors._id: 67e4b035af7d0551dc377e15
paper_authors.name: Wufei Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T10:15:53.618Z
paper_authors._id: 625f81afe1994410eef1c36a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1650426282769-noauth.jpeg
paper_authors.fullname: Wufei Ma
paper_authors.type: user
paper_authors.user: wufeim
paper_authors._id: 67e4b035af7d0551dc377e16
paper_authors.name: Alan Yuille
paper_title: DINeMo: Learning Neural Mesh Models with no 3D Annotations
paper_summary: Category-level 3D/6D pose estimation is a crucial step towards comprehensive
  3D scene understanding, which would enable a broad range of applications in
  robotics and embodied AI. Recent works explored neural mesh models that
  approach a range of 2D and 3D tasks from an analysis-by-synthesis perspective.
  Despite the largely enhanced robustness to partial occlusion and domain shifts,
  these methods depended heavily on 3D annotations for part-contrastive learning,
  which confines them to a narrow set of categories and hinders efficient
  scaling. In this work, we present DINeMo, a novel neural mesh model that is
  trained with no 3D annotations by leveraging pseudo-correspondence obtained
  from large visual foundation models. We adopt a bidirectional
  pseudo-correspondence generation method, which produce pseudo correspondence
  utilize both local appearance features and global context information.
  Experimental results on car datasets demonstrate that our DINeMo outperforms
  previous zero- and few-shot 3D pose estimation by a wide margin, narrowing the
  gap with fully-supervised methods by 67.3%. Our DINeMo also scales effectively
  and efficiently when incorporating more unlabeled images during training, which
  demonstrate the advantages over supervised learning methods that rely on 3D
  annotations. Our project page is available at
  https://analysis-by-synthesis.github.io/DINeMo/.
paper_ai_summary: DINeMo, a novel neural mesh model, achieves superior zero- and few-shot 3D pose estimation by generating pseudo-correspondences through large visual foundation models, outperforming supervised methods.
paper_ai_keywords: neural mesh models analysis-by-synthesis pseudo-correspondence visual foundation models bidirectional pseudo-correspondence 3D pose estimation 3D annotations car datasets zero-shot few-shot supervised learning