Distributed NeRF Learning for Collaborative Multi-Robot Perception
<|reference_start|>Distributed NeRF Learning for Collaborative Multi-Robot Perception: Effective environment perception is crucial for enabling downstream robotic applications. Individual robotic agents often face occlusion and limited visibility issues, whereas multi-agent systems can offer a more comprehensive mapping of the environment, quicker coverage, and increased fault tolerance. In this paper, we propose a collaborative multi-agent perception system where agents collectively learn a neural radiance field (NeRF) from posed RGB images to represent a scene. Each agent processes its local sensory data and shares only its learned NeRF model with other agents, reducing communication overhead. Given NeRF's low memory footprint, this approach is well-suited for robotic systems with limited bandwidth, where transmitting all raw data is impractical. Our distributed learning framework ensures consistency across agents' local NeRF models, enabling convergence to a unified scene representation. We show the effectiveness of our method through an extensive set of experiments on datasets containing challenging real-world scenes, achieving performance comparable to centralized mapping of the environment where data is sent to a central server for processing. Additionally, we find that multi-agent learning provides regularization benefits, improving geometric consistency in scenarios with sparse input views. We show that in such scenarios, multi-agent mapping can even outperform centralized training.<|reference_end|>
@article{zhao2024distributed,
    title={Distributed NeRF Learning for Collaborative Multi-Robot Perception},
    author={Hongrui Zhao, Boris Ivanovic, Negar Mehr},
    journal={arXiv preprint arXiv:2409.20289},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.20289},
    primaryClass={cs.RO cs.CV cs.LG}
}