Customizable Perturbation Synthesis for Robust SLAM Benchmarking
<|reference_start|>Customizable Perturbation Synthesis for Robust SLAM Benchmarking: Robustness is a crucial factor for the successful deployment of robots in unstructured environments, particularly in the domain of Simultaneous Localization and Mapping (SLAM). Simulation-based benchmarks have emerged as a highly scalable approach for robustness evaluation compared to real-world data collection. However, crafting a challenging and controllable noisy world with diverse perturbations remains relatively under-explored. To this end, we propose a novel, customizable pipeline for noisy data synthesis, aimed at assessing the resilience of multi-modal SLAM models against various perturbations. This pipeline incorporates customizable hardware setups, software components, and perturbed environments. In particular, we introduce comprehensive perturbation taxonomy along with a perturbation composition toolbox, allowing the transformation of clean simulations into challenging noisy environments. Utilizing the pipeline, we instantiate the Robust-SLAM benchmark, which includes diverse perturbation types, to evaluate the risk tolerance of existing advanced multi-modal SLAM models. Our extensive analysis uncovers the susceptibilities of existing SLAM models to real-world disturbance, despite their demonstrated accuracy in standard benchmarks. Our perturbation synthesis toolbox, SLAM robustness evaluation pipeline, and Robust-SLAM benchmark will be made publicly available at https://github.com/Xiaohao-Xu/SLAM-under-Perturbation/.<|reference_end|>
@article{xu2024customizable,
    title={Customizable Perturbation Synthesis for Robust SLAM Benchmarking},
    author={Xiaohao Xu, Tianyi Zhang, Sibo Wang, Xiang Li, Yongqi Chen, Ye Li,
  Bhiksha Raj, Matthew Johnson-Roberson, Xiaonan Huang},
    journal={arXiv preprint arXiv:2402.08125},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.08125},
    primaryClass={cs.RO cs.AI cs.CV cs.MM}
}