title: SANeRF-HQ: Segment Anything for NeRF in High Quality
summary: Recently, the Segment Anything Model (SAM) has showcased remarkable
  capabilities of zero-shot segmentation, while NeRF (Neural Radiance Fields) has
  gained popularity as a method for various 3D problems beyond novel view
  synthesis. Though there exist initial attempts to incorporate these two methods
  into 3D segmentation, they face the challenge of accurately and consistently
  segmenting objects in complex scenarios. In this paper, we introduce the
  Segment Anything for NeRF in High Quality (SANeRF-HQ) to achieve high quality
  3D segmentation of any object in a given scene. SANeRF-HQ utilizes SAM for
  open-world object segmentation guided by user-supplied prompts, while
  leveraging NeRF to aggregate information from different viewpoints. To overcome
  the aforementioned challenges, we employ density field and RGB similarity to
  enhance the accuracy of segmentation boundary during the aggregation.
  Emphasizing on segmentation accuracy, we evaluate our method quantitatively on
  multiple NeRF datasets where high-quality ground-truths are available or
  manually annotated. SANeRF-HQ shows a significant quality improvement over
  previous state-of-the-art methods in NeRF object segmentation, provides higher
  flexibility for object localization, and enables more consistent object
  segmentation across multiple views. Additional information can be found at
  https://lyclyc52.github.io/SANeRF-HQ/.
paper_authors._id: 656e9ec298c38db7db03d282
paper_authors.name: Yichen Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-08T10:20:00.253Z
paper_authors._id: 6572dcc6bbd6664053b1fa6b
paper_authors.avatarUrl: /avatars/aba29efd00bc41f14ce422f7807cd2c3.svg
paper_authors.fullname: Liu Yichen
paper_authors.type: user
paper_authors.user: lyclyc52
paper_authors._id: 656e9ec298c38db7db03d283
paper_authors.name: Benran Hu
paper_authors._id: 656e9ec298c38db7db03d284
paper_authors.name: Chi-Keung Tang
paper_authors._id: 656e9ec298c38db7db03d285
paper_authors.name: Yu-Wing Tai
paper_title: SANeRF-HQ: Segment Anything for NeRF in High Quality
paper_summary: Recently, the Segment Anything Model (SAM) has showcased remarkable
  capabilities of zero-shot segmentation, while NeRF (Neural Radiance Fields) has
  gained popularity as a method for various 3D problems beyond novel view
  synthesis. Though there exist initial attempts to incorporate these two methods
  into 3D segmentation, they face the challenge of accurately and consistently
  segmenting objects in complex scenarios. In this paper, we introduce the
  Segment Anything for NeRF in High Quality (SANeRF-HQ) to achieve high quality
  3D segmentation of any object in a given scene. SANeRF-HQ utilizes SAM for
  open-world object segmentation guided by user-supplied prompts, while
  leveraging NeRF to aggregate information from different viewpoints. To overcome
  the aforementioned challenges, we employ density field and RGB similarity to
  enhance the accuracy of segmentation boundary during the aggregation.
  Emphasizing on segmentation accuracy, we evaluate our method quantitatively on
  multiple NeRF datasets where high-quality ground-truths are available or
  manually annotated. SANeRF-HQ shows a significant quality improvement over
  previous state-of-the-art methods in NeRF object segmentation, provides higher
  flexibility for object localization, and enables more consistent object
  segmentation across multiple views. Additional information can be found at
  https://lyclyc52.github.io/SANeRF-HQ/.
paper_ai_summary: A novel method, SANeRF-HQ, combines Segment Anything Model (SAM) and NeRF for high-quality 3D object segmentation across multiple views using density field and RGB similarity.
paper_ai_keywords: Segment Anything Model SAM NeRF Neural Radiance Fields 3D segmentation density field RGB similarity open-world object segmentation novel view synthesis object localization