title: SAMPart3D: Segment Any Part in 3D Objects
summary: 3D part segmentation is a crucial and challenging task in 3D perception,
  playing a vital role in applications such as robotics, 3D generation, and 3D
  editing. Recent methods harness the powerful Vision Language Models (VLMs) for
  2D-to-3D knowledge distillation, achieving zero-shot 3D part segmentation.
  However, these methods are limited by their reliance on text prompts, which
  restricts the scalability to large-scale unlabeled datasets and the flexibility
  in handling part ambiguities. In this work, we introduce SAMPart3D, a scalable
  zero-shot 3D part segmentation framework that segments any 3D object into
  semantic parts at multiple granularities, without requiring predefined part
  label sets as text prompts. For scalability, we use text-agnostic vision
  foundation models to distill a 3D feature extraction backbone, allowing scaling
  to large unlabeled 3D datasets to learn rich 3D priors. For flexibility, we
  distill scale-conditioned part-aware 3D features for 3D part segmentation at
  multiple granularities. Once the segmented parts are obtained from the
  scale-conditioned part-aware 3D features, we use VLMs to assign semantic labels
  to each part based on the multi-view renderings. Compared to previous methods,
  our SAMPart3D can scale to the recent large-scale 3D object dataset Objaverse
  and handle complex, non-ordinary objects. Additionally, we contribute a new 3D
  part segmentation benchmark to address the lack of diversity and complexity of
  objects and parts in existing benchmarks. Experiments show that our SAMPart3D
  significantly outperforms existing zero-shot 3D part segmentation methods, and
  can facilitate various applications such as part-level editing and interactive
  segmentation.
paper_authors._id: 67341bc8bd4f7e2dfe198a26
paper_authors.name: Yunhan Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:28:49.914Z
paper_authors._id: 64b4eecf2fc8324fcb63b404
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b4eecf2fc8324fcb63b404/zGYqYVB4-o-GBMybJ8CDA.png
paper_authors.fullname: Yunhan Yang
paper_authors.type: user
paper_authors.user: yhyang-myron
paper_authors._id: 67341bc8bd4f7e2dfe198a27
paper_authors.name: Yukun Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T17:25:45.766Z
paper_authors._id: 638ee900ee7e45e0474a5712
paper_authors.avatarUrl: /avatars/eadb5ae2fc92bd9af5516acbc8f1bdf0.svg
paper_authors.fullname: Yukun Huang
paper_authors.type: user
paper_authors.user: KevinHuang
paper_authors._id: 67341bc8bd4f7e2dfe198a28
paper_authors.name: Yuan-Chen Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:32:02.999Z
paper_authors._id: 6346aaa3f06b237ba4e297b0
paper_authors.avatarUrl: /avatars/5acb986e993eab1461200f3e9d99d022.svg
paper_authors.fullname: Yuan-Chen Guo
paper_authors.type: user
paper_authors.user: bennyguo
paper_authors._id: 67341bc8bd4f7e2dfe198a29
paper_authors.name: Liangjun Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:32:13.410Z
paper_authors._id: 664095e66246363bdd6acd7c
paper_authors.avatarUrl: /avatars/f3d3a4d19f1f9397bcb9cb07afcafcfb.svg
paper_authors.fullname: luliangjun
paper_authors.type: user
paper_authors.user: kurtlu
paper_authors._id: 67341bc8bd4f7e2dfe198a2a
paper_authors.name: Xiaoyang Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:32:37.270Z
paper_authors._id: 643e5d6a1d0e956d94bb3608
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643e5d6a1d0e956d94bb3608/mTnu7Dts2RmDklHlp9Gqu.jpeg
paper_authors.fullname: Xiaoyang Wu
paper_authors.type: user
paper_authors.user: Gofinge
paper_authors._id: 67341bc8bd4f7e2dfe198a2b
paper_authors.name: Edmund Y. Lam
paper_authors._id: 67341bc8bd4f7e2dfe198a2c
paper_authors.name: Yan-Pei Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:32:52.905Z
paper_authors._id: 638066faf022c8a5803f7eb8
paper_authors.avatarUrl: /avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
paper_authors.fullname: Yanpei Cao
paper_authors.type: user
paper_authors.user: pookiefoof
paper_authors._id: 67341bc8bd4f7e2dfe198a2d
paper_authors.name: Xihui Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:25:20.354Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_title: SAMPart3D: Segment Any Part in 3D Objects
paper_summary: 3D part segmentation is a crucial and challenging task in 3D perception,
  playing a vital role in applications such as robotics, 3D generation, and 3D
  editing. Recent methods harness the powerful Vision Language Models (VLMs) for
  2D-to-3D knowledge distillation, achieving zero-shot 3D part segmentation.
  However, these methods are limited by their reliance on text prompts, which
  restricts the scalability to large-scale unlabeled datasets and the flexibility
  in handling part ambiguities. In this work, we introduce SAMPart3D, a scalable
  zero-shot 3D part segmentation framework that segments any 3D object into
  semantic parts at multiple granularities, without requiring predefined part
  label sets as text prompts. For scalability, we use text-agnostic vision
  foundation models to distill a 3D feature extraction backbone, allowing scaling
  to large unlabeled 3D datasets to learn rich 3D priors. For flexibility, we
  distill scale-conditioned part-aware 3D features for 3D part segmentation at
  multiple granularities. Once the segmented parts are obtained from the
  scale-conditioned part-aware 3D features, we use VLMs to assign semantic labels
  to each part based on the multi-view renderings. Compared to previous methods,
  our SAMPart3D can scale to the recent large-scale 3D object dataset Objaverse
  and handle complex, non-ordinary objects. Additionally, we contribute a new 3D
  part segmentation benchmark to address the lack of diversity and complexity of
  objects and parts in existing benchmarks. Experiments show that our SAMPart3D
  significantly outperforms existing zero-shot 3D part segmentation methods, and
  can facilitate various applications such as part-level editing and interactive
  segmentation.
paper_ai_keywords: Vision Language Models (VLMs) 3D part segmentation 3D feature extraction backbone Scale-conditioned part-aware 3D features Multi-view renderings 3D priors 3D perception 3D datasets Objaverse 3D part segmentation benchmark Part-level editing Interactive segmentation