title: Segment Any 3D Gaussians
summary: Interactive 3D segmentation in radiance fields is an appealing task since its
  importance in 3D scene understanding and manipulation. However, existing
  methods face challenges in either achieving fine-grained, multi-granularity
  segmentation or contending with substantial computational overhead, inhibiting
  real-time interaction. In this paper, we introduce Segment Any 3D GAussians
  (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a
  2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent
  breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D
  segmentation results generated by the segmentation foundation model into 3D
  Gaussian point features through well-designed contrastive training. Evaluation
  on existing benchmarks demonstrates that SAGA can achieve competitive
  performance with state-of-the-art methods. Moreover, SAGA achieves
  multi-granularity segmentation and accommodates various prompts, including
  points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation
  within milliseconds, achieving nearly 1000x acceleration compared to previous
  SOTA. The project page is at https://jumpat.github.io/SAGA.
paper_authors._id: 656ea0467112260ee143e9aa
paper_authors.name: Jiazhong Cen
paper_authors._id: 656ea0467112260ee143e9ab
paper_authors.name: Jiemin Fang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:02:56.172Z
paper_authors._id: 652cf49cc6857682d3168f5a
paper_authors.avatarUrl: /avatars/9ccffa04832429aad37320a301ea8e36.svg
paper_authors.fullname: Jiemin Fang
paper_authors.type: user
paper_authors.user: JieminFang
paper_authors._id: 656ea0467112260ee143e9ac
paper_authors.name: Chen Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-19T09:35:26.592Z
paper_authors._id: 6440a1c0e44f30a72325e9ea
paper_authors.avatarUrl: /avatars/b76919a4aeaedc8c735a1bcfc50136c1.svg
paper_authors.fullname: Chen Yang
paper_authors.type: user
paper_authors.user: chen-yang-ln
paper_authors._id: 656ea0467112260ee143e9ad
paper_authors.name: Lingxi Xie
paper_authors._id: 656ea0467112260ee143e9ae
paper_authors.name: Xiaopeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:03:48.052Z
paper_authors._id: 640bea9cb4f73086cddbe0a6
paper_authors.avatarUrl: /avatars/ae460dd971de1d04660b266b22f81f9d.svg
paper_authors.fullname: Xiaopeng Zhang
paper_authors.type: user
paper_authors.user: xpzhang
paper_authors._id: 656ea0467112260ee143e9af
paper_authors.name: Wei Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-06T10:25:12.898Z
paper_authors._id: 656fc3b124baa875504bdf92
paper_authors.avatarUrl: /avatars/951194df5e212a2e62cc8fe77cf47b69.svg
paper_authors.fullname: WEI SHEN
paper_authors.type: user
paper_authors.user: wshen10
paper_authors._id: 656ea0467112260ee143e9b0
paper_authors.name: Qi Tian
paper_title: Segment Any 3D Gaussians
paper_summary: Interactive 3D segmentation in radiance fields is an appealing task since its
  importance in 3D scene understanding and manipulation. However, existing
  methods face challenges in either achieving fine-grained, multi-granularity
  segmentation or contending with substantial computational overhead, inhibiting
  real-time interaction. In this paper, we introduce Segment Any 3D GAussians
  (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a
  2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent
  breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D
  segmentation results generated by the segmentation foundation model into 3D
  Gaussian point features through well-designed contrastive training. Evaluation
  on existing benchmarks demonstrates that SAGA can achieve competitive
  performance with state-of-the-art methods. Moreover, SAGA achieves
  multi-granularity segmentation and accommodates various prompts, including
  points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation
  within milliseconds, achieving nearly 1000x acceleration compared to previous
  SOTA. The project page is at https://jumpat.github.io/SAGA.
paper_ai_keywords: 3D segmentation radiance fields comprehensive, multi-granularity segmentation computational overhead real-time interaction Segment Any 3D GAussians (SAGA) 3D Gaussian Splatting (3DGS) 2D segmentation foundation model embedding multi-granularity segmentation 2D segmentation results contrast training evaluation benchmarks state-of-the-art (SOTA) prompts scribbles 2D masks milliseconds acceleration