Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics
<|reference_start|>Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics: Swarm navigation in cluttered environments is a grand challenge in robotics. This work combines deep learning with first-principle physics through differentiable simulation to enable autonomous navigation of multiple aerial robots through complex environments at high speed. Our approach optimizes a neural network control policy directly by backpropagating loss gradients through the robot simulation using a simple point-mass physics model and a depth rendering engine. Despite this simplicity, our method excels in challenging tasks for both multi-agent and single-agent applications with zero-shot sim-to-real transfer. In multi-agent scenarios, our system demonstrates self-organized behavior, enabling autonomous coordination without communication or centralized planning - an achievement not seen in existing traditional or learning-based methods. In single-agent scenarios, our system achieves a 90% success rate in navigating through complex environments, significantly surpassing the 60% success rate of the previous state-of-the-art approach. Our system can operate without state estimation and adapt to dynamic obstacles. In real-world forest environments, it navigates at speeds up to 20 m/s, doubling the speed of previous imitation learning-based solutions. Notably, all these capabilities are deployed on a budget-friendly $21 computer, costing less than 5% of a GPU-equipped board used in existing systems. Video demonstrations are available at https://youtu.be/LKg9hJqc2cc.<|reference_end|>
@article{zhang2024back,
    title={Back to Newton's Laws: Learning Vision-based Agile Flight via
  Differentiable Physics},
    author={Yuang Zhang, Yu Hu, Yunlong Song, Danping Zou, Weiyao Lin},
    journal={arXiv preprint arXiv:2407.10648},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.10648},
    primaryClass={cs.RO}
}