  Camera and radar sensors have significant advantages in cost, reliability,
and maintenance compared to LiDAR. Existing fusion methods often fuse the
outputs of single modalities at the result-level, called the late fusion
strategy. This can benefit from using off-the-shelf single sensor detection
algorithms, but late fusion cannot fully exploit the complementary properties
of sensors, thus having limited performance despite the huge potential of
camera-radar fusion. Here we propose a novel proposal-level early fusion
approach that effectively exploits both spatial and contextual properties of
camera and radar for 3D object detection. Our fusion framework first associates
image proposal with radar points in the polar coordinate system to efficiently
handle the discrepancy between the coordinate system and spatial properties.
Using this as a first stage, following consecutive cross-attention based
feature fusion layers adaptively exchange spatio-contextual information between
camera and radar, leading to a robust and attentive fusion. Our camera-radar
fusion approach achieves the state-of-the-art 41.1% mAP and 52.3% NDS on the
nuScenes test set, which is 8.7 and 10.8 points higher than the camera-only
baseline, as well as yielding competitive performance on the LiDAR method.

Computer Vision and Pattern Recognition;Artificial Intelligence;Robotics