title: 3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and
    Box-Focused Sampling for 3D Object Detection
summary: Neural Radiance Fields (NeRF) are widely used for novel-view synthesis and
  have been adapted for 3D Object Detection (3DOD), offering a promising approach
  to 3DOD through view-synthesis representation. However, NeRF faces inherent
  limitations: (i) limited representational capacity for 3DOD due to its implicit
  nature, and (ii) slow rendering speeds. Recently, 3D Gaussian Splatting (3DGS)
  has emerged as an explicit 3D representation that addresses these limitations.
  Inspired by these advantages, this paper introduces 3DGS into 3DOD for the
  first time, identifying two main challenges: (i) Ambiguous spatial distribution
  of Gaussian blobs: 3DGS primarily relies on 2D pixel-level supervision,
  resulting in unclear 3D spatial distribution of Gaussian blobs and poor
  differentiation between objects and background, which hinders 3DOD; (ii)
  Excessive background blobs: 2D images often include numerous background pixels,
  leading to densely reconstructed 3DGS with many noisy Gaussian blobs
  representing the background, negatively affecting detection. To tackle the
  challenge (i), we leverage the fact that 3DGS reconstruction is derived from 2D
  images, and propose an elegant and efficient solution by incorporating 2D
  Boundary Guidance to significantly enhance the spatial distribution of Gaussian
  blobs, resulting in clearer differentiation between objects and their
  background. To address the challenge (ii), we propose a Box-Focused Sampling
  strategy using 2D boxes to generate object probability distribution in 3D
  spaces, allowing effective probabilistic sampling in 3D to retain more object
  blobs and reduce noisy background blobs. Benefiting from our designs, our
  3DGS-DET significantly outperforms the SOTA NeRF-based method, NeRF-Det,
  achieving improvements of +6.6 on mAP@0.25 and +8.1 on mAP@0.5 for the ScanNet
  dataset, and impressive +31.5 on mAP@0.25 for the ARKITScenes dataset.
paper_authors._id: 66fe459a609fa71aa6f715e7
paper_authors.name: Yang Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:58:43.253Z
paper_authors._id: 647d893bcfca67bc50f8974f
paper_authors.avatarUrl: /avatars/fab275f968b7d648dbdf8485c9279fb6.svg
paper_authors.fullname: Yang Cao
paper_authors.type: user
paper_authors.user: YangCaoCS
paper_authors._id: 66fe459a609fa71aa6f715e8
paper_authors.name: Yuanliang Jv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:58:10.852Z
paper_authors._id: 646f4cf46b3df773a2c93fd0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646f4cf46b3df773a2c93fd0/8SGp5O6tmbd26Oxte9Gpy.jpeg
paper_authors.fullname: Yuanliang Ju
paper_authors.type: user
paper_authors.user: Yuanliang
paper_authors._id: 66fe459a609fa71aa6f715e9
paper_authors.name: Dan Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T16:33:14.238Z
paper_authors._id: 66feab48651e00e22f33222e
paper_authors.avatarUrl: /avatars/7344377e2c796c7ec85194bb2fc78521.svg
paper_authors.fullname: Dan Xu
paper_authors.type: user
paper_authors.user: danxuhk
paper_title: 3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and
    Box-Focused Sampling for 3D Object Detection
paper_summary: Neural Radiance Fields (NeRF) are widely used for novel-view synthesis and
  have been adapted for 3D Object Detection (3DOD), offering a promising approach
  to 3DOD through view-synthesis representation. However, NeRF faces inherent
  limitations: (i) limited representational capacity for 3DOD due to its implicit
  nature, and (ii) slow rendering speeds. Recently, 3D Gaussian Splatting (3DGS)
  has emerged as an explicit 3D representation that addresses these limitations.
  Inspired by these advantages, this paper introduces 3DGS into 3DOD for the
  first time, identifying two main challenges: (i) Ambiguous spatial distribution
  of Gaussian blobs: 3DGS primarily relies on 2D pixel-level supervision,
  resulting in unclear 3D spatial distribution of Gaussian blobs and poor
  differentiation between objects and background, which hinders 3DOD; (ii)
  Excessive background blobs: 2D images often include numerous background pixels,
  leading to densely reconstructed 3DGS with many noisy Gaussian blobs
  representing the background, negatively affecting detection. To tackle the
  challenge (i), we leverage the fact that 3DGS reconstruction is derived from 2D
  images, and propose an elegant and efficient solution by incorporating 2D
  Boundary Guidance to significantly enhance the spatial distribution of Gaussian
  blobs, resulting in clearer differentiation between objects and their
  background. To address the challenge (ii), we propose a Box-Focused Sampling
  strategy using 2D boxes to generate object probability distribution in 3D
  spaces, allowing effective probabilistic sampling in 3D to retain more object
  blobs and reduce noisy background blobs. Benefiting from our designs, our
  3DGS-DET significantly outperforms the SOTA NeRF-based method, NeRF-Det,
  achieving improvements of +6.6 on mAP@0.25 and +8.1 on mAP@0.5 for the ScanNet
  dataset, and impressive +31.5 on mAP@0.25 for the ARKITScenes dataset.
paper_ai_summary: 3D Gaussian Splatting is introduced for 3D object detection, improving NeRF's limitations through 2D Boundary Guidance and Box-Focused Sampling strategies, achieving superior performance on ScanNet and ARKITScenes datasets.
paper_ai_keywords: Neural Radiance Fields 3D Object Detection view-synthesis representation implicit nature rendering speeds 3D Gaussian Splatting explicit 3D representation Gaussian blobs 2D pixel-level supervision 2D Boundary Guidance Box-Focused Sampling object probability distribution mAP@0.25 mAP@0.5 ScanNet ARKITScenes