title: Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding
summary: In this paper, we propose an efficient multi-level convolution architecture
  for 3D visual grounding. Conventional methods are difficult to meet the
  requirements of real-time inference due to the two-stage or point-based
  architecture. Inspired by the success of multi-level fully sparse convolutional
  architecture in 3D object detection, we aim to build a new 3D visual grounding
  framework following this technical route. However, as in 3D visual grounding
  task the 3D scene representation should be deeply interacted with text
  features, sparse convolution-based architecture is inefficient for this
  interaction due to the large amount of voxel features. To this end, we propose
  text-guided pruning (TGP) and completion-based addition (CBA) to deeply fuse 3D
  scene representation and text features in an efficient way by gradual region
  pruning and target completion. Specifically, TGP iteratively sparsifies the 3D
  scene representation and thus efficiently interacts the voxel features with
  text features by cross-attention. To mitigate the affect of pruning on delicate
  geometric information, CBA adaptively fixes the over-pruned region by voxel
  completion with negligible computational overhead. Compared with previous
  single-stage methods, our method achieves top inference speed and surpasses
  previous fastest method by 100\% FPS. Our method also achieves state-of-the-art
  accuracy even compared with two-stage methods, with +1.13 lead of Acc@0.5 on
  ScanRefer, and +2.6 and +3.2 leads on NR3D and SR3D respectively. The code
  is available at
  https://github.com/GWxuan/TSP3D{https://github.com/GWxuan/TSP3D}.
paper_authors._id: 67b346bab6c58a3e0a26190a
paper_authors.name: Wenxuan Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:32:20.352Z
paper_authors._id: 67b2cf648a276e7b4856e307
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/lHepLDwyYeQC4IQhVKgz6.png
paper_authors.fullname: Wenxuan Guo
paper_authors.type: user
paper_authors.user: gwx22
paper_authors._id: 67b346bab6c58a3e0a26190b
paper_authors.name: Xiuwei Xu
paper_authors._id: 67b346bab6c58a3e0a26190c
paper_authors.name: Ziwei Wang
paper_authors._id: 67b346bab6c58a3e0a26190d
paper_authors.name: Jianjiang Feng
paper_authors._id: 67b346bab6c58a3e0a26190e
paper_authors.name: Jie Zhou
paper_authors._id: 67b346bab6c58a3e0a26190f
paper_authors.name: Jiwen Lu
paper_title: Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding
paper_summary: In this paper, we propose an efficient multi-level convolution architecture
  for 3D visual grounding. Conventional methods are difficult to meet the
  requirements of real-time inference due to the two-stage or point-based
  architecture. Inspired by the success of multi-level fully sparse convolutional
  architecture in 3D object detection, we aim to build a new 3D visual grounding
  framework following this technical route. However, as in 3D visual grounding
  task the 3D scene representation should be deeply interacted with text
  features, sparse convolution-based architecture is inefficient for this
  interaction due to the large amount of voxel features. To this end, we propose
  text-guided pruning (TGP) and completion-based addition (CBA) to deeply fuse 3D
  scene representation and text features in an efficient way by gradual region
  pruning and target completion. Specifically, TGP iteratively sparsifies the 3D
  scene representation and thus efficiently interacts the voxel features with
  text features by cross-attention. To mitigate the affect of pruning on delicate
  geometric information, CBA adaptively fixes the over-pruned region by voxel
  completion with negligible computational overhead. Compared with previous
  single-stage methods, our method achieves top inference speed and surpasses
  previous fastest method by 100\% FPS. Our method also achieves state-of-the-art
  accuracy even compared with two-stage methods, with +1.13 lead of Acc@0.5 on
  ScanRefer, and +2.6 and +3.2 leads on NR3D and SR3D respectively. The code
  is available at
  https://github.com/GWxuan/TSP3D{https://github.com/GWxuan/TSP3D}.
paper_ai_keywords: multi-level convolution architecture 3D visual grounding two-stage architecture point-based architecture multi-level fully sparse convolutional architecture 3D object detection 3D scene representation text features text-guided pruning (TGP) completion-based addition (CBA) cross-attention voxel completion ScanRefer NR3D SR3D