title: SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Manners
summary: We introduce SAM2Point, a preliminary exploration adapting Segment Anything
  Model 2 (SAM 2) for zero-shot and promptable 3D segmentation. SAM2Point
  interprets any 3D data as a series of multi-directional videos, and leverages
  SAM 2 for 3D-space segmentation, without further training or 2D-3D projection.
  Our framework supports various prompt types, including 3D points, boxes, and
  masks, and can generalize across diverse scenarios, such as 3D objects, indoor
  scenes, outdoor environments, and raw sparse LiDAR. Demonstrations on multiple
  3D datasets, e.g., Objaverse, S3DIS, ScanNet, Semantic3D, and KITTI, highlight
  the robust generalization capabilities of SAM2Point. To our best knowledge, we
  present the most faithful implementation of SAM in 3D, which may serve as a
  starting point for future research in promptable 3D segmentation. Online Demo:
  https://huggingface.co/spaces/ZiyuG/SAM2Point . Code:
  https://github.com/ZiyuGuo99/SAM2Point .
paper_authors._id: 66d11cfebd2cd7aee09aba4b
paper_authors.name: Ziyu Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T12:47:12.670Z
paper_authors._id: 647d9ab61a1fcad2fdbf2d3d
paper_authors.avatarUrl: /avatars/48c8aeae8979d2c87df8bde922437d62.svg
paper_authors.fullname: Ziyu Guo
paper_authors.type: user
paper_authors.user: ZiyuG
paper_authors._id: 66d11cfebd2cd7aee09aba4c
paper_authors.name: Renrui Zhang
paper_authors._id: 66d11cfebd2cd7aee09aba4d
paper_authors.name: Xiangyang Zhu
paper_authors._id: 66d11cfebd2cd7aee09aba4e
paper_authors.name: Chengzhuo Tong
paper_authors._id: 66d11cfebd2cd7aee09aba4f
paper_authors.name: Peng Gao
paper_authors._id: 66d11cfebd2cd7aee09aba50
paper_authors.name: Chunyuan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T12:48:39.941Z
paper_authors._id: 62aba526cae4462c0c6caa0f
paper_authors.avatarUrl: /avatars/430560ec2c2547f819225769ab432f30.svg
paper_authors.fullname: Chunyuan Li
paper_authors.type: user
paper_authors.user: Chunyuan24
paper_authors._id: 66d11cfebd2cd7aee09aba51
paper_authors.name: Pheng-Ann Heng
paper_title: SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Manners
paper_summary: We introduce SAM2Point, a preliminary exploration adapting Segment Anything
  Model 2 (SAM 2) for zero-shot and promptable 3D segmentation. SAM2Point
  interprets any 3D data as a series of multi-directional videos, and leverages
  SAM 2 for 3D-space segmentation, without further training or 2D-3D projection.
  Our framework supports various prompt types, including 3D points, boxes, and
  masks, and can generalize across diverse scenarios, such as 3D objects, indoor
  scenes, outdoor environments, and raw sparse LiDAR. Demonstrations on multiple
  3D datasets, e.g., Objaverse, S3DIS, ScanNet, Semantic3D, and KITTI, highlight
  the robust generalization capabilities of SAM2Point. To our best knowledge, we
  present the most faithful implementation of SAM in 3D, which may serve as a
  starting point for future research in promptable 3D segmentation. Online Demo:
  https://huggingface.co/spaces/ZiyuG/SAM2Point . Code:
  https://github.com/ZiyuGuo99/SAM2Point .
paper_ai_keywords: SAM2Point Segment Anything Model 2 (SAM 2) zero-shot segmentation promptable segmentation 3D segmentation multi-directional videos 3D-space segmentation 3D points 3D boxes 3D masks dual framework Objaverse S3DIS ScanNet Semantic3D KITTI fidelity in 3D promptable 3D segmentation