SANeRF-HQ: Segment Anything for NeRF in High Quality
Recently, the Segment Anything Model (SAM) has showcased remarkable
capabilities of zero-shot segmentation, while NeRF (Neural Radiance Fields) has
gained popularity as a method for various 3D problems beyond novel view
synthesis. Though there exist initial attempts to incorporate these two methods
into 3D segmentation, they face the challenge of accurately and consistently
segmenting objects in complex scenarios. In this paper, we introduce the
Segment Anything for NeRF in High Quality (SANeRF-HQ) to achieve high quality
3D segmentation of any object in a given scene. SANeRF-HQ utilizes SAM for
open-world object segmentation guided by user-supplied prompts, while
leveraging NeRF to aggregate information from different viewpoints. To overcome
the aforementioned challenges, we employ density field and RGB similarity to
enhance the accuracy of segmentation boundary during the aggregation.
Emphasizing on segmentation accuracy, we evaluate our method quantitatively on
multiple NeRF datasets where high-quality ground-truths are available or
manually annotated. SANeRF-HQ shows a significant quality improvement over
previous state-of-the-art methods in NeRF object segmentation, provides higher
flexibility for object localization, and enables more consistent object
segmentation across multiple views. Additional information can be found at
https://lyclyc52.github.io/SANeRF-HQ/.
656e9ec298c38db7db03d282
Yichen Liu
claimed_verified
2023-12-08T10:20:00.253Z
6572dcc6bbd6664053b1fa6b
/avatars/aba29efd00bc41f14ce422f7807cd2c3.svg
Liu Yichen
user
lyclyc52
656e9ec298c38db7db03d283
Benran Hu
656e9ec298c38db7db03d284
Chi-Keung Tang
656e9ec298c38db7db03d285
Yu-Wing Tai
SANeRF-HQ: Segment Anything for NeRF in High Quality
Recently, the Segment Anything Model (SAM) has showcased remarkable
capabilities of zero-shot segmentation, while NeRF (Neural Radiance Fields) has
gained popularity as a method for various 3D problems beyond novel view
synthesis. Though there exist initial attempts to incorporate these two methods
into 3D segmentation, they face the challenge of accurately and consistently
segmenting objects in complex scenarios. In this paper, we introduce the
Segment Anything for NeRF in High Quality (SANeRF-HQ) to achieve high quality
3D segmentation of any object in a given scene. SANeRF-HQ utilizes SAM for
open-world object segmentation guided by user-supplied prompts, while
leveraging NeRF to aggregate information from different viewpoints. To overcome
the aforementioned challenges, we employ density field and RGB similarity to
enhance the accuracy of segmentation boundary during the aggregation.
Emphasizing on segmentation accuracy, we evaluate our method quantitatively on
multiple NeRF datasets where high-quality ground-truths are available or
manually annotated. SANeRF-HQ shows a significant quality improvement over
previous state-of-the-art methods in NeRF object segmentation, provides higher
flexibility for object localization, and enables more consistent object
segmentation across multiple views. Additional information can be found at
https://lyclyc52.github.io/SANeRF-HQ/.
A novel method, SANeRF-HQ, combines Segment Anything Model (SAM) and NeRF for high-quality 3D object segmentation across multiple views using density field and RGB similarity.
Segment Anything Model SAM NeRF Neural Radiance Fields 3D segmentation density field RGB similarity open-world object segmentation novel view synthesis object localization