SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation
<|reference_start|>SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation: Autonomous self-improving robots that interact and improve with experience are key to the real-world deployment of robotic systems. In this paper, we propose an online learning method, SELFI, that leverages online robot experience to rapidly fine-tune pre-trained control policies efficiently. SELFI applies online model-free reinforcement learning on top of offline model-based learning to bring out the best parts of both learning paradigms. Specifically, SELFI stabilizes the online learning process by incorporating the same model-based learning objective from offline pre-training into the Q-values learned with online model-free reinforcement learning. We evaluate SELFI in multiple real-world environments and report improvements in terms of collision avoidance, as well as more socially compliant behavior, measured by a human user study. SELFI enables us to quickly learn useful robotic behaviors with less human interventions such as pre-emptive behavior for the pedestrians, collision avoidance for small and transparent objects, and avoiding travel on uneven floor surfaces. We provide supplementary videos to demonstrate the performance of our fine-tuned policy on our project page.<|reference_end|>
@article{hirose2024selfi:,
    title={SELFI: Autonomous Self-Improvement with Reinforcement Learning for
  Social Navigation},
    author={Noriaki Hirose, Dhruv Shah, Kyle Stachowicz, Ajay Sridhar and Sergey
  Levine},
    journal={arXiv preprint arXiv:2403.00991},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.00991},
    primaryClass={cs.RO cs.CV cs.LG}
}