Learning H-Infinity Locomotion Control
Stable locomotion in precipitous environments is an essential capability of
quadruped robots, demanding the ability to resist various external
disturbances. However, recent learning-based policies only use basic domain
randomization to improve the robustness of learned policies, which cannot
guarantee that the robot has adequate disturbance resistance capabilities. In
this paper, we propose to model the learning process as an adversarial
interaction between the actor and a newly introduced disturber and ensure their
optimization with H_{infty} constraint. In contrast to the actor that
maximizes the discounted overall reward, the disturber is responsible for
generating effective external forces and is optimized by maximizing the error
between the task reward and its oracle, i.e., "cost" in each iteration. To keep
joint optimization between the actor and the disturber stable, our H_{infty}
constraint mandates the bound of ratio between the cost to the intensity of the
external forces. Through reciprocal interaction throughout the training phase,
the actor can acquire the capability to navigate increasingly complex physical
disturbances. We verify the robustness of our approach on quadrupedal
locomotion tasks with Unitree Aliengo robot, and also a more challenging task
with Unitree A1 robot, where the quadruped is expected to perform locomotion
merely on its hind legs as if it is a bipedal robot. The simulated quantitative
results show improvement against baselines, demonstrating the effectiveness of
the method and each design choice. On the other hand, real-robot experiments
qualitatively exhibit how robust the policy is when interfering with various
disturbances on various terrains, including stairs, high platforms, slopes, and
slippery terrains. All code, checkpoints, and real-world deployment guidance
will be made public.
662728b497eadbff1683227e
Junfeng Long
claimed_verified
2024-04-23T09:09:46.015Z
6627741bef7dd857cc5d1504
https://cdn-avatars.huggingface.co/v1/production/uploads/6627741bef7dd857cc5d1504/xPAW_7jl31Oy8pPx6skzF.jpeg
Junfeng Long
user
Junfeng-Long
662728b497eadbff1683227f
Wenye Yu
admin_assigned
2024-04-23T09:09:55.150Z
659987c7d108fc317f829c26
https://cdn-avatars.huggingface.co/v1/production/uploads/659987c7d108fc317f829c26/BS5cOTM6TW9zZpxE4c_hH.jpeg
Wenye Yu
user
Virlus
662728b497eadbff16832280
Quanyi Li
662728b497eadbff16832281
Zirui Wang
662728b497eadbff16832282
Dahua Lin
admin_assigned
2024-04-23T09:10:32.180Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
662728b497eadbff16832283
Jiangmiao Pang
admin_assigned
2024-04-23T09:10:38.105Z
65783ee6ee33d547aecc3ffc
https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
Jiangmiao Pang
user
Jiangmiao
Learning H-Infinity Locomotion Control
Stable locomotion in precipitous environments is an essential capability of
quadruped robots, demanding the ability to resist various external
disturbances. However, recent learning-based policies only use basic domain
randomization to improve the robustness of learned policies, which cannot
guarantee that the robot has adequate disturbance resistance capabilities. In
this paper, we propose to model the learning process as an adversarial
interaction between the actor and a newly introduced disturber and ensure their
optimization with H_{infty} constraint. In contrast to the actor that
maximizes the discounted overall reward, the disturber is responsible for
generating effective external forces and is optimized by maximizing the error
between the task reward and its oracle, i.e., "cost" in each iteration. To keep
joint optimization between the actor and the disturber stable, our H_{infty}
constraint mandates the bound of ratio between the cost to the intensity of the
external forces. Through reciprocal interaction throughout the training phase,
the actor can acquire the capability to navigate increasingly complex physical
disturbances. We verify the robustness of our approach on quadrupedal
locomotion tasks with Unitree Aliengo robot, and also a more challenging task
with Unitree A1 robot, where the quadruped is expected to perform locomotion
merely on its hind legs as if it is a bipedal robot. The simulated quantitative
results show improvement against baselines, demonstrating the effectiveness of
the method and each design choice. On the other hand, real-robot experiments
qualitatively exhibit how robust the policy is when interfering with various
disturbances on various terrains, including stairs, high platforms, slopes, and
slippery terrains. All code, checkpoints, and real-world deployment guidance
will be made public.
A novel adversarial training method with an $H_{\infty}$ constraint enhances quadruped robots' robustness to external disturbances, demonstrated through simulations and real-world deployments on Unitree Aliengo and A1 robots.
adversarial interaction $H_{\infty}$ constraint actor disturber discounted overall reward task reward oracle cost external forces reciprocal interaction quadrupedal locomotion Unitree Aliengo Unitree A1 real-robot experiments robust policy various terrains stairs high platforms slopes slippery terrains