An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement
<|reference_start|>An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement: Learning-based methods have improved locomotion skills of quadruped robots through deep reinforcement learning. However, the sim-to-real gap and low sample efficiency still limit the skill transfer. To address this issue, we propose an efficient model-based learning framework that combines a world model with a policy network. We train a differentiable world model to predict future states and use it to directly supervise a Variational Autoencoder (VAE)-based policy network to imitate real animal behaviors. This significantly reduces the need for real interaction data and allows for rapid policy updates. We also develop a high-level network to track diverse commands and trajectories. Our simulated results show a tenfold sample efficiency increase compared to reinforcement learning methods such as PPO. In real-world testing, our policy achieves proficient command-following performance with only a two-minute data collection period and generalizes well to new speeds and paths.<|reference_end|>
@article{shi2024an,
    title={An Efficient Model-Based Approach on Learning Agile Motor Skills without
  Reinforcement},
    author={Haojie Shi, Tingguang Li, Qingxu Zhu, Jiapeng Sheng, Lei Han and Max
  Q.-H. Meng},
    journal={arXiv preprint arXiv:2403.01962},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.01962},
    primaryClass={cs.RO}
}