title: Sim-to-Real Reinforcement Learning for Vision-Based Dexterous
    Manipulation on Humanoids
summary: Reinforcement learning has delivered promising results in achieving human- or
  even superhuman-level capabilities across diverse problem domains, but success
  in dexterous robot manipulation remains limited. This work investigates the key
  challenges in applying reinforcement learning to solve a collection of
  contact-rich manipulation tasks on a humanoid embodiment. We introduce novel
  techniques to overcome the identified challenges with empirical validation. Our
  main contributions include an automated real-to-sim tuning module that brings
  the simulated environment closer to the real world, a generalized reward design
  scheme that simplifies reward engineering for long-horizon contact-rich
  manipulation tasks, a divide-and-conquer distillation process that improves the
  sample efficiency of hard-exploration problems while maintaining sim-to-real
  performance, and a mixture of sparse and dense object representations to bridge
  the sim-to-real perception gap. We show promising results on three humanoid
  dexterous manipulation tasks, with ablation studies on each technique. Our work
  presents a successful approach to learning humanoid dexterous manipulation
  using sim-to-real reinforcement learning, achieving robust generalization and
  high performance without the need for human demonstration.
paper_authors._id: 67c51d36c830dcb76bbb5994
paper_authors.name: Toru Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-03T16:07:25.709Z
paper_authors._id: 65e8b34632f166badb8d893a
paper_authors.avatarUrl: /avatars/a55da1d08dc1104e6c539cd3f1ef1ebe.svg
paper_authors.fullname: T
paper_authors.type: user
paper_authors.user: toruowo
paper_authors._id: 67c51d36c830dcb76bbb5995
paper_authors.name: Kartik Sachdev
paper_authors._id: 67c51d36c830dcb76bbb5996
paper_authors.name: Linxi Fan
paper_authors._id: 67c51d36c830dcb76bbb5997
paper_authors.name: Jitendra Malik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:36:34.177Z
paper_authors._id: 65369a95605a07338de78ab0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sGFjOjLT2akN-sn5beVWL.jpeg
paper_authors.fullname: Jitendra Malik 
paper_authors.type: user
paper_authors.user: jitendra1995
paper_authors._id: 67c51d36c830dcb76bbb5998
paper_authors.name: Yuke Zhu
paper_title: Sim-to-Real Reinforcement Learning for Vision-Based Dexterous
    Manipulation on Humanoids
paper_summary: Reinforcement learning has delivered promising results in achieving human- or
  even superhuman-level capabilities across diverse problem domains, but success
  in dexterous robot manipulation remains limited. This work investigates the key
  challenges in applying reinforcement learning to solve a collection of
  contact-rich manipulation tasks on a humanoid embodiment. We introduce novel
  techniques to overcome the identified challenges with empirical validation. Our
  main contributions include an automated real-to-sim tuning module that brings
  the simulated environment closer to the real world, a generalized reward design
  scheme that simplifies reward engineering for long-horizon contact-rich
  manipulation tasks, a divide-and-conquer distillation process that improves the
  sample efficiency of hard-exploration problems while maintaining sim-to-real
  performance, and a mixture of sparse and dense object representations to bridge
  the sim-to-real perception gap. We show promising results on three humanoid
  dexterous manipulation tasks, with ablation studies on each technique. Our work
  presents a successful approach to learning humanoid dexterous manipulation
  using sim-to-real reinforcement learning, achieving robust generalization and
  high performance without the need for human demonstration.
paper_ai_keywords: reinforcement learning humanoid embodiment contact-rich manipulation tasks automated real-to-sim tuning generalized reward design reward engineering long-horizon tasks divide-and-conquer distillation sample efficiency hard-exploration sim-to-real performance sparse object representations dense object representations sim-to-real perception gap humanoid dexterous manipulation robust generalization