Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding
In this paper, we propose an efficient multi-level convolution architecture
for 3D visual grounding. Conventional methods are difficult to meet the
requirements of real-time inference due to the two-stage or point-based
architecture. Inspired by the success of multi-level fully sparse convolutional
architecture in 3D object detection, we aim to build a new 3D visual grounding
framework following this technical route. However, as in 3D visual grounding
task the 3D scene representation should be deeply interacted with text
features, sparse convolution-based architecture is inefficient for this
interaction due to the large amount of voxel features. To this end, we propose
text-guided pruning (TGP) and completion-based addition (CBA) to deeply fuse 3D
scene representation and text features in an efficient way by gradual region
pruning and target completion. Specifically, TGP iteratively sparsifies the 3D
scene representation and thus efficiently interacts the voxel features with
text features by cross-attention. To mitigate the affect of pruning on delicate
geometric information, CBA adaptively fixes the over-pruned region by voxel
completion with negligible computational overhead. Compared with previous
single-stage methods, our method achieves top inference speed and surpasses
previous fastest method by 100\% FPS. Our method also achieves state-of-the-art
accuracy even compared with two-stage methods, with +1.13 lead of Acc@0.5 on
ScanRefer, and +2.6 and +3.2 leads on NR3D and SR3D respectively. The code
is available at
https://github.com/GWxuan/TSP3D{https://github.com/GWxuan/TSP3D}.
67b346bab6c58a3e0a26190a
Wenxuan Guo
claimed_verified
2025-02-18T09:32:20.352Z
67b2cf648a276e7b4856e307
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/lHepLDwyYeQC4IQhVKgz6.png
Wenxuan Guo
user
gwx22
67b346bab6c58a3e0a26190b
Xiuwei Xu
67b346bab6c58a3e0a26190c
Ziwei Wang
67b346bab6c58a3e0a26190d
Jianjiang Feng
67b346bab6c58a3e0a26190e
Jie Zhou
67b346bab6c58a3e0a26190f
Jiwen Lu
Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding
In this paper, we propose an efficient multi-level convolution architecture
for 3D visual grounding. Conventional methods are difficult to meet the
requirements of real-time inference due to the two-stage or point-based
architecture. Inspired by the success of multi-level fully sparse convolutional
architecture in 3D object detection, we aim to build a new 3D visual grounding
framework following this technical route. However, as in 3D visual grounding
task the 3D scene representation should be deeply interacted with text
features, sparse convolution-based architecture is inefficient for this
interaction due to the large amount of voxel features. To this end, we propose
text-guided pruning (TGP) and completion-based addition (CBA) to deeply fuse 3D
scene representation and text features in an efficient way by gradual region
pruning and target completion. Specifically, TGP iteratively sparsifies the 3D
scene representation and thus efficiently interacts the voxel features with
text features by cross-attention. To mitigate the affect of pruning on delicate
geometric information, CBA adaptively fixes the over-pruned region by voxel
completion with negligible computational overhead. Compared with previous
single-stage methods, our method achieves top inference speed and surpasses
previous fastest method by 100\% FPS. Our method also achieves state-of-the-art
accuracy even compared with two-stage methods, with +1.13 lead of Acc@0.5 on
ScanRefer, and +2.6 and +3.2 leads on NR3D and SR3D respectively. The code
is available at
https://github.com/GWxuan/TSP3D{https://github.com/GWxuan/TSP3D}.
A multi-level convolution architecture with text-guided pruning and completion-based addition achieves real-time inference speed and state-of-the-art accuracy for 3D visual grounding.
multi-level convolution architecture 3D visual grounding fully sparse convolutional architecture 3D object detection text features voxels text-guided pruning cross-attention completion-based addition voxel completion top inference speed state-of-the-art accuracy ScanRefer NR3D SR3D