LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera
<|reference_start|>LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera: Semantic Scene Completion (SSC) is pivotal in autonomous driving perception, frequently confronted with the complexities of weather and illumination changes. The long-term strategy involves fusing multi-modal information to bolster the system's robustness. Radar, increasingly utilized for 3D target detection, is gradually replacing LiDAR in autonomous driving applications, offering a robust sensing alternative. In this paper, we focus on the potential of 3D radar in semantic scene completion, pioneering cross-modal refinement techniques for improved robustness against weather and illumination changes, and enhancing SSC performance.Regarding model architecture, we propose a three-stage tight fusion approach on BEV to realize a fusion framework for point clouds and images. Based on this foundation, we designed three cross-modal distillation modules-CMRD, BRD, and PDD. Our approach enhances the performance in both radar-only (R-LiCROcc) and radar-camera (RC-LiCROcc) settings by distilling to them the rich semantic and structural information of the fused features of LiDAR and camera. Finally, our LC-Fusion (teacher model), R-LiCROcc and RC-LiCROcc achieve the best performance on the nuScenes-Occupancy dataset, with mIOU exceeding the baseline by 22.9%, 44.1%, and 15.5%, respectively. The project page is available at https://hr-zju.github.io/LiCROcc/.<|reference_end|>
@article{ma2024licrocc:,
    title={LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using
  LiDAR and Camera},
    author={Yukai Ma, Jianbiao Mei, Xuemeng Yang, Licheng Wen, Weihua Xu,
  Jiangning Zhang, Botian Shi, Yong Liu, Xingxing Zuo},
    journal={arXiv preprint arXiv:2407.16197},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.16197},
    primaryClass={cs.CV cs.RO}
}