Sim-to-Real Reinforcement Learning for Vision-Based Dexterous
  Manipulation on Humanoids
Reinforcement learning has delivered promising results in achieving human- or
even superhuman-level capabilities across diverse problem domains, but success
in dexterous robot manipulation remains limited. This work investigates the key
challenges in applying reinforcement learning to solve a collection of
contact-rich manipulation tasks on a humanoid embodiment. We introduce novel
techniques to overcome the identified challenges with empirical validation. Our
main contributions include an automated real-to-sim tuning module that brings
the simulated environment closer to the real world, a generalized reward design
scheme that simplifies reward engineering for long-horizon contact-rich
manipulation tasks, a divide-and-conquer distillation process that improves the
sample efficiency of hard-exploration problems while maintaining sim-to-real
performance, and a mixture of sparse and dense object representations to bridge
the sim-to-real perception gap. We show promising results on three humanoid
dexterous manipulation tasks, with ablation studies on each technique. Our work
presents a successful approach to learning humanoid dexterous manipulation
using sim-to-real reinforcement learning, achieving robust generalization and
high performance without the need for human demonstration.
67c51d36c830dcb76bbb5994
Toru Lin
claimed_verified
2025-03-03T16:07:25.709Z
65e8b34632f166badb8d893a
/avatars/a55da1d08dc1104e6c539cd3f1ef1ebe.svg
T
user
toruowo
67c51d36c830dcb76bbb5995
Kartik Sachdev
67c51d36c830dcb76bbb5996
Linxi Fan
67c51d36c830dcb76bbb5997
Jitendra Malik
admin_assigned
2025-03-03T09:36:34.177Z
65369a95605a07338de78ab0
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sGFjOjLT2akN-sn5beVWL.jpeg
Jitendra Malik 
user
jitendra1995
67c51d36c830dcb76bbb5998
Yuke Zhu
Sim-to-Real Reinforcement Learning for Vision-Based Dexterous
  Manipulation on Humanoids
Reinforcement learning has delivered promising results in achieving human- or
even superhuman-level capabilities across diverse problem domains, but success
in dexterous robot manipulation remains limited. This work investigates the key
challenges in applying reinforcement learning to solve a collection of
contact-rich manipulation tasks on a humanoid embodiment. We introduce novel
techniques to overcome the identified challenges with empirical validation. Our
main contributions include an automated real-to-sim tuning module that brings
the simulated environment closer to the real world, a generalized reward design
scheme that simplifies reward engineering for long-horizon contact-rich
manipulation tasks, a divide-and-conquer distillation process that improves the
sample efficiency of hard-exploration problems while maintaining sim-to-real
performance, and a mixture of sparse and dense object representations to bridge
the sim-to-real perception gap. We show promising results on three humanoid
dexterous manipulation tasks, with ablation studies on each technique. Our work
presents a successful approach to learning humanoid dexterous manipulation
using sim-to-real reinforcement learning, achieving robust generalization and
high performance without the need for human demonstration.
This work introduces techniques to enhance reinforcement learning for humanoid dexterous manipulation through real-to-sim tuning, generalized reward design, divide-and-conquer distillation, and mixed object representations, achieving promising performance on manipulation tasks.
reinforcement learning dexterous manipulation contact-rich tasks humanoid embodiment real-to-sim tuning generalized reward design divide-and-conquer distillation sparse and dense object representations