RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection
Jingtong Yue Zhiwei Lin Xin Lin Xiaoyu Zhou Xiangtai Li Lu Qi Yongtao Wang Ming-Hsuan Yang
While recent low-cost radar-camera approaches have shown promising results in
multi-modal 3D object detection, both sensors face challenges from environmen-
tal and intrinsic disturbances. Poor lighting or adverse weather conditions de-
grade camera performance, while radar suffers from noise and positional ambigu-
ity. Achieving robust radar-camera 3D object detection requires consistent perfor-
mance across varying conditions, a topic that has not yet been fully explored. In
this work, we first conduct a systematic analysis of robustness in radar-camera de-
tection on five kinds of noises and propose RobuRCDet, a robust object detection
model in bird’s eye view (BEV). Specifically, we design a 3D Gaussian Expan-
sion (3DGE) module to mitigate inaccuracies in radar points, including position,
Radar Cross-Section (RCS), and velocity. The 3DGE uses RCS and velocity priors
to generate a deformable kernel map and variance for kernel size adjustment and
value distribution. Additionally, we introduce a weather-adaptive fusion module,
which adaptively fuses radar and camera features based on camera signal confi-
dence. Extensive experiments on the popular benchmark, nuScenes, show that
our RobuRCDet achieves competitive results in regular and noisy conditions. The
source codes and trained models will be made available.
3D Vision， Radar Camera 3D Object Detection