title: Learning H-Infinity Locomotion Control
summary: Stable locomotion in precipitous environments is an essential capability of
  quadruped robots, demanding the ability to resist various external
  disturbances. However, recent learning-based policies only use basic domain
  randomization to improve the robustness of learned policies, which cannot
  guarantee that the robot has adequate disturbance resistance capabilities. In
  this paper, we propose to model the learning process as an adversarial
  interaction between the actor and a newly introduced disturber and ensure their
  optimization with H_{infty} constraint. In contrast to the actor that
  maximizes the discounted overall reward, the disturber is responsible for
  generating effective external forces and is optimized by maximizing the error
  between the task reward and its oracle, i.e., "cost" in each iteration. To keep
  joint optimization between the actor and the disturber stable, our H_{infty}
  constraint mandates the bound of ratio between the cost to the intensity of the
  external forces. Through reciprocal interaction throughout the training phase,
  the actor can acquire the capability to navigate increasingly complex physical
  disturbances. We verify the robustness of our approach on quadrupedal
  locomotion tasks with Unitree Aliengo robot, and also a more challenging task
  with Unitree A1 robot, where the quadruped is expected to perform locomotion
  merely on its hind legs as if it is a bipedal robot. The simulated quantitative
  results show improvement against baselines, demonstrating the effectiveness of
  the method and each design choice. On the other hand, real-robot experiments
  qualitatively exhibit how robust the policy is when interfering with various
  disturbances on various terrains, including stairs, high platforms, slopes, and
  slippery terrains. All code, checkpoints, and real-world deployment guidance
  will be made public.
paper_authors._id: 662728b497eadbff1683227e
paper_authors.name: Junfeng Long
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-23T09:09:46.015Z
paper_authors._id: 6627741bef7dd857cc5d1504
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6627741bef7dd857cc5d1504/xPAW_7jl31Oy8pPx6skzF.jpeg
paper_authors.fullname: Junfeng Long
paper_authors.type: user
paper_authors.user: Junfeng-Long
paper_authors._id: 662728b497eadbff1683227f
paper_authors.name: Wenye Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T09:09:55.150Z
paper_authors._id: 659987c7d108fc317f829c26
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/659987c7d108fc317f829c26/BS5cOTM6TW9zZpxE4c_hH.jpeg
paper_authors.fullname: Wenye Yu
paper_authors.type: user
paper_authors.user: Virlus
paper_authors._id: 662728b497eadbff16832280
paper_authors.name: Quanyi Li
paper_authors._id: 662728b497eadbff16832281
paper_authors.name: Zirui Wang
paper_authors._id: 662728b497eadbff16832282
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T09:10:32.180Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 662728b497eadbff16832283
paper_authors.name: Jiangmiao Pang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T09:10:38.105Z
paper_authors._id: 65783ee6ee33d547aecc3ffc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
paper_authors.fullname: Jiangmiao Pang
paper_authors.type: user
paper_authors.user: Jiangmiao
paper_title: Learning H-Infinity Locomotion Control
paper_summary: Stable locomotion in precipitous environments is an essential capability of
  quadruped robots, demanding the ability to resist various external
  disturbances. However, recent learning-based policies only use basic domain
  randomization to improve the robustness of learned policies, which cannot
  guarantee that the robot has adequate disturbance resistance capabilities. In
  this paper, we propose to model the learning process as an adversarial
  interaction between the actor and a newly introduced disturber and ensure their
  optimization with H_{infty} constraint. In contrast to the actor that
  maximizes the discounted overall reward, the disturber is responsible for
  generating effective external forces and is optimized by maximizing the error
  between the task reward and its oracle, i.e., "cost" in each iteration. To keep
  joint optimization between the actor and the disturber stable, our H_{infty}
  constraint mandates the bound of ratio between the cost to the intensity of the
  external forces. Through reciprocal interaction throughout the training phase,
  the actor can acquire the capability to navigate increasingly complex physical
  disturbances. We verify the robustness of our approach on quadrupedal
  locomotion tasks with Unitree Aliengo robot, and also a more challenging task
  with Unitree A1 robot, where the quadruped is expected to perform locomotion
  merely on its hind legs as if it is a bipedal robot. The simulated quantitative
  results show improvement against baselines, demonstrating the effectiveness of
  the method and each design choice. On the other hand, real-robot experiments
  qualitatively exhibit how robust the policy is when interfering with various
  disturbances on various terrains, including stairs, high platforms, slopes, and
  slippery terrains. All code, checkpoints, and real-world deployment guidance
  will be made public.
paper_ai_summary: A novel adversarial training method with an $H_{\infty}$ constraint enhances quadruped robots' robustness to external disturbances, demonstrated through simulations and real-world deployments on Unitree Aliengo and A1 robots.
paper_ai_keywords: adversarial interaction $H_{\infty}$ constraint actor disturber discounted overall reward task reward oracle cost external forces reciprocal interaction quadrupedal locomotion Unitree Aliengo Unitree A1 real-robot experiments robust policy various terrains stairs high platforms slopes slippery terrains