title: Perpetual Humanoid Control for Real-time Simulated Avatars
summary: We present a physics-based humanoid controller that achieves high-fidelity
  motion imitation and fault-tolerant behavior in the presence of noisy input
  (e.g. pose estimates from video or generated from language) and unexpected
  falls. Our controller scales up to learning ten thousand motion clips without
  using any external stabilizing forces and learns to naturally recover from
  fail-state. Given reference motion, our controller can perpetually control
  simulated avatars without requiring resets. At its core, we propose the
  progressive multiplicative control policy (PMCP), which dynamically allocates
  new network capacity to learn harder and harder motion sequences. PMCP allows
  efficient scaling for learning from large-scale motion databases and adding new
  tasks, such as fail-state recovery, without catastrophic forgetting. We
  demonstrate the effectiveness of our controller by using it to imitate noisy
  poses from video-based pose estimators and language-based motion generators in
  a live and real-time multi-person avatar use case.
paper_authors._id: 645dcb6403fc86c46b42313d
paper_authors.name: Zhengyi Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-14T17:51:29.818Z
paper_authors._id: 645eaf37f0f92653b9f2ba77
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VM-hXdknuRfeK3BD9FbyR.png
paper_authors.fullname: Luo
paper_authors.type: user
paper_authors.user: Zhengyiluo
paper_authors._id: 645dcb6403fc86c46b42313e
paper_authors.name: Jinkun Cao
paper_authors._id: 645dcb6403fc86c46b42313f
paper_authors.name: Alexander Winkler
paper_authors._id: 645dcb6403fc86c46b423140
paper_authors.name: Kris Kitani
paper_authors._id: 645dcb6403fc86c46b423141
paper_authors.name: Weipeng Xu
paper_title: Perpetual Humanoid Control for Real-time Simulated Avatars
paper_summary: We present a physics-based humanoid controller that achieves high-fidelity
  motion imitation and fault-tolerant behavior in the presence of noisy input
  (e.g. pose estimates from video or generated from language) and unexpected
  falls. Our controller scales up to learning ten thousand motion clips without
  using any external stabilizing forces and learns to naturally recover from
  fail-state. Given reference motion, our controller can perpetually control
  simulated avatars without requiring resets. At its core, we propose the
  progressive multiplicative control policy (PMCP), which dynamically allocates
  new network capacity to learn harder and harder motion sequences. PMCP allows
  efficient scaling for learning from large-scale motion databases and adding new
  tasks, such as fail-state recovery, without catastrophic forgetting. We
  demonstrate the effectiveness of our controller by using it to imitate noisy
  poses from video-based pose estimators and language-based motion generators in
  a live and real-time multi-person avatar use case.
paper_ai_keywords: physics-based humanoid controller high-fidelity motion imitation fault-tolerant behavior pose estimates video generated from language unexpected falls multiplicative control policy (PMCP) dynamic network capacity allocation large-scale databases fail-state recovery recovery catastrophic forgetting video-based pose estimators language-based motion generators multi-person avatar use case