SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Manners
We introduce SAM2Point, a preliminary exploration adapting Segment Anything
Model 2 (SAM 2) for zero-shot and promptable 3D segmentation. SAM2Point
interprets any 3D data as a series of multi-directional videos, and leverages
SAM 2 for 3D-space segmentation, without further training or 2D-3D projection.
Our framework supports various prompt types, including 3D points, boxes, and
masks, and can generalize across diverse scenarios, such as 3D objects, indoor
scenes, outdoor environments, and raw sparse LiDAR. Demonstrations on multiple
3D datasets, e.g., Objaverse, S3DIS, ScanNet, Semantic3D, and KITTI, highlight
the robust generalization capabilities of SAM2Point. To our best knowledge, we
present the most faithful implementation of SAM in 3D, which may serve as a
starting point for future research in promptable 3D segmentation. Online Demo:
https://huggingface.co/spaces/ZiyuG/SAM2Point . Code:
https://github.com/ZiyuGuo99/SAM2Point .
66d11cfebd2cd7aee09aba4b
Ziyu Guo
admin_assigned
2024-08-30T12:47:12.670Z
647d9ab61a1fcad2fdbf2d3d
/avatars/48c8aeae8979d2c87df8bde922437d62.svg
Ziyu Guo
user
ZiyuG
66d11cfebd2cd7aee09aba4c
Renrui Zhang
66d11cfebd2cd7aee09aba4d
Xiangyang Zhu
66d11cfebd2cd7aee09aba4e
Chengzhuo Tong
66d11cfebd2cd7aee09aba4f
Peng Gao
66d11cfebd2cd7aee09aba50
Chunyuan Li
admin_assigned
2024-08-30T12:48:39.941Z
62aba526cae4462c0c6caa0f
/avatars/430560ec2c2547f819225769ab432f30.svg
Chunyuan Li
user
Chunyuan24
66d11cfebd2cd7aee09aba51
Pheng-Ann Heng
SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Manners
We introduce SAM2Point, a preliminary exploration adapting Segment Anything
Model 2 (SAM 2) for zero-shot and promptable 3D segmentation. SAM2Point
interprets any 3D data as a series of multi-directional videos, and leverages
SAM 2 for 3D-space segmentation, without further training or 2D-3D projection.
Our framework supports various prompt types, including 3D points, boxes, and
masks, and can generalize across diverse scenarios, such as 3D objects, indoor
scenes, outdoor environments, and raw sparse LiDAR. Demonstrations on multiple
3D datasets, e.g., Objaverse, S3DIS, ScanNet, Semantic3D, and KITTI, highlight
the robust generalization capabilities of SAM2Point. To our best knowledge, we
present the most faithful implementation of SAM in 3D, which may serve as a
starting point for future research in promptable 3D segmentation. Online Demo:
https://huggingface.co/spaces/ZiyuG/SAM2Point . Code:
https://github.com/ZiyuGuo99/SAM2Point .
SAM2Point leverages SAM 2 for zero-shot promptable 3D segmentation by interpreting 3D data as multi-directional videos, supporting various prompts and scenarios without additional training or 2D-3D projection.
Segment Anything Model 2 SAM 2 zero-shot promptable 3D segmentation multi-directional videos 3D points boxes masks Objaverse S3DIS ScanNet Semantic3D KITTI promptable 3D segmentation