Real-time Multi-view Omnidirectional Depth Estimation System for Robots and Autonomous Driving on Real Scenes
<|reference_start|>Real-time Multi-view Omnidirectional Depth Estimation System for Robots and Autonomous Driving on Real Scenes: Omnidirectional Depth Estimation has broad application prospects in fields such as robotic navigation and autonomous driving. In this paper, we propose a robotic prototype system and corresponding algorithm designed to validate omnidirectional depth estimation for navigation and obstacle avoidance in real-world scenarios for both robots and vehicles. The proposed HexaMODE system captures 360$^\circ$ depth maps using six surrounding arranged fisheye cameras. We introduce a combined spherical sweeping method and optimize the model architecture for proposed RtHexa-OmniMVS algorithm to achieve real-time omnidirectional depth estimation. To ensure high accuracy, robustness, and generalization in real-world environments, we employ a teacher-student self-training strategy, utilizing large-scale unlabeled real-world data for model training. The proposed algorithm demonstrates high accuracy in various complex real-world scenarios, both indoors and outdoors, achieving an inference speed of 15 fps on edge computing platforms.<|reference_end|>
@article{li2024real-time,
    title={Real-time Multi-view Omnidirectional Depth Estimation System for Robots
  and Autonomous Driving on Real Scenes},
    author={Ming Li, Xiong Yang, Chaofan Wu, Jiaheng Li, Pinzhi Wang, Xuejiao Hu,
  Sidan Du and Yang Li},
    journal={arXiv preprint arXiv:2409.07843},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.07843},
    primaryClass={cs.CV cs.RO}
}