title: SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model
summary: With the development of large language models, many remarkable linguistic
  systems like ChatGPT have thrived and achieved astonishing success on many
  tasks, showing the incredible power of foundation models. In the spirit of
  unleashing the capability of foundation models on vision tasks, the Segment
  Anything Model (SAM), a vision foundation model for image segmentation, has
  been proposed recently and presents strong zero-shot ability on many downstream
  2D tasks. However, whether SAM can be adapted to 3D vision tasks has yet to be
  explored, especially 3D object detection. With this inspiration, we explore
  adapting the zero-shot ability of SAM to 3D object detection in this paper. We
  propose a SAM-powered BEV processing pipeline to detect objects and get
  promising results on the large-scale Waymo open dataset. As an early attempt,
  our method takes a step toward 3D object detection with vision foundation
  models and presents the opportunity to unleash their power on 3D vision tasks.
  The code is released at https://github.com/DYZhang09/SAM3D.
paper_authors._id: 647eaa0ae4d52fe0e021df92
paper_authors.name: Dingyuan Zhang
paper_authors._id: 647eaa0ae4d52fe0e021df93
paper_authors.name: Dingkang Liang
paper_authors._id: 647eaa0ae4d52fe0e021df94
paper_authors.name: Hongcheng Yang
paper_authors._id: 647eaa0ae4d52fe0e021df95
paper_authors.name: Zhikang Zou
paper_authors._id: 647eaa0ae4d52fe0e021df96
paper_authors.name: Xiaoqing Ye
paper_authors._id: 647eaa0ae4d52fe0e021df97
paper_authors.name: Zhe Liu
paper_authors._id: 647eaa0ae4d52fe0e021df98
paper_authors.name: Xiang Bai
paper_title: SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model
paper_summary: With the development of large language models, many remarkable linguistic
  systems like ChatGPT have thrived and achieved astonishing success on many
  tasks, showing the incredible power of foundation models. In the spirit of
  unleashing the capability of foundation models on vision tasks, the Segment
  Anything Model (SAM), a vision foundation model for image segmentation, has
  been proposed recently and presents strong zero-shot ability on many downstream
  2D tasks. However, whether SAM can be adapted to 3D vision tasks has yet to be
  explored, especially 3D object detection. With this inspiration, we explore
  adapting the zero-shot ability of SAM to 3D object detection in this paper. We
  propose a SAM-powered BEV processing pipeline to detect objects and get
  promising results on the large-scale Waymo open dataset. As an early attempt,
  our method takes a step toward 3D object detection with vision foundation
  models and presents the opportunity to unleash their power on 3D vision tasks.
  The code is released at https://github.com/DYZhang09/SAM3D.
paper_ai_summary: A SAM-powered BEV processing pipeline demonstrates promising results for 3D object detection on the Waymo open dataset, extending the zero-shot capabilities of vision foundation models to 3D tasks.
paper_ai_keywords: Segment Anything Model (SAM) vision foundation model image segmentation zero-shot ability 3D object detection BEV processing pipeline Waymo open dataset