Segment Any 3D Gaussians
Interactive 3D segmentation in radiance fields is an appealing task since its
importance in 3D scene understanding and manipulation. However, existing
methods face challenges in either achieving fine-grained, multi-granularity
segmentation or contending with substantial computational overhead, inhibiting
real-time interaction. In this paper, we introduce Segment Any 3D GAussians
(SAGA), a novel 3D interactive segmentation approach that seamlessly blends a
2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent
breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D
segmentation results generated by the segmentation foundation model into 3D
Gaussian point features through well-designed contrastive training. Evaluation
on existing benchmarks demonstrates that SAGA can achieve competitive
performance with state-of-the-art methods. Moreover, SAGA achieves
multi-granularity segmentation and accommodates various prompts, including
points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation
within milliseconds, achieving nearly 1000x acceleration compared to previous
SOTA. The project page is at https://jumpat.github.io/SAGA.
656ea0467112260ee143e9aa
Jiazhong Cen
656ea0467112260ee143e9ab
Jiemin Fang
admin_assigned
2023-12-05T09:02:56.172Z
652cf49cc6857682d3168f5a
/avatars/9ccffa04832429aad37320a301ea8e36.svg
Jiemin Fang
user
JieminFang
656ea0467112260ee143e9ac
Chen Yang
claimed_verified
2024-02-19T09:35:26.592Z
6440a1c0e44f30a72325e9ea
/avatars/b76919a4aeaedc8c735a1bcfc50136c1.svg
Chen Yang
user
chen-yang-ln
656ea0467112260ee143e9ad
Lingxi Xie
656ea0467112260ee143e9ae
Xiaopeng Zhang
admin_assigned
2023-12-05T09:03:48.052Z
640bea9cb4f73086cddbe0a6
/avatars/ae460dd971de1d04660b266b22f81f9d.svg
Xiaopeng Zhang
user
xpzhang
656ea0467112260ee143e9af
Wei Shen
claimed_verified
2023-12-06T10:25:12.898Z
656fc3b124baa875504bdf92
/avatars/951194df5e212a2e62cc8fe77cf47b69.svg
WEI SHEN
user
wshen10
656ea0467112260ee143e9b0
Qi Tian
Segment Any 3D Gaussians
Interactive 3D segmentation in radiance fields is an appealing task since its
importance in 3D scene understanding and manipulation. However, existing
methods face challenges in either achieving fine-grained, multi-granularity
segmentation or contending with substantial computational overhead, inhibiting
real-time interaction. In this paper, we introduce Segment Any 3D GAussians
(SAGA), a novel 3D interactive segmentation approach that seamlessly blends a
2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent
breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D
segmentation results generated by the segmentation foundation model into 3D
Gaussian point features through well-designed contrastive training. Evaluation
on existing benchmarks demonstrates that SAGA can achieve competitive
performance with state-of-the-art methods. Moreover, SAGA achieves
multi-granularity segmentation and accommodates various prompts, including
points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation
within milliseconds, achieving nearly 1000x acceleration compared to previous
SOTA. The project page is at https://jumpat.github.io/SAGA.
SAGA efficiently combines 2D segmentation models with 3D Gaussian Splatting for real-time, multi-granularity 3D interactive segmentation with low computational overhead.
radiance fields 3D Gaussian Splatting 3DGS 2D segmentation foundation model contrastive training