ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via
  Residual Learning
Human hands play a central role in interacting, motivating increasing
research in dexterous robotic manipulation. Data-driven embodied AI algorithms
demand precise, large-scale, human-like manipulation sequences, which are
challenging to obtain with conventional reinforcement learning or real-world
teleoperation. To address this, we introduce ManipTrans, a novel two-stage
method for efficiently transferring human bimanual skills to dexterous robotic
hands in simulation. ManipTrans first pre-trains a generalist trajectory
imitator to mimic hand motion, then fine-tunes a specific residual module under
interaction constraints, enabling efficient learning and accurate execution of
complex bimanual tasks. Experiments show that ManipTrans surpasses
state-of-the-art methods in success rate, fidelity, and efficiency. Leveraging
ManipTrans, we transfer multiple hand-object datasets to robotic hands,
creating DexManipNet, a large-scale dataset featuring previously unexplored
tasks like pen capping and bottle unscrewing. DexManipNet comprises 3.3K
episodes of robotic manipulation and is easily extensible, facilitating further
policy training for dexterous hands and enabling real-world deployments.
67ec97f65d9c75ff46de2974
Kailin Li
67ec97f65d9c75ff46de2975
Puhao Li
admin_assigned
2025-04-02T12:29:34.965Z
63d4e83913ae45b7807acbb1
/avatars/4b8778ed239b64de5f8781f0ba57168b.svg
Puhao Li
user
puhao
67ec97f65d9c75ff46de2976
Tengyu Liu
admin_assigned
2025-04-02T12:29:41.351Z
634379a81bdd3dfa55dcbe82
https://cdn-avatars.huggingface.co/v1/production/uploads/1665366425318-noauth.png
Tengyu Liu
user
oxrider
67ec97f65d9c75ff46de2977
Yuyang Li
admin_assigned
2025-04-02T12:29:48.257Z
637cef70b8e573d75beb5748
https://cdn-avatars.huggingface.co/v1/production/uploads/637cef70b8e573d75beb5748/hMl_pKUO3BanR47RQqN6o.png
Yuyang Li
user
aidenli
67ec97f65d9c75ff46de2978
Siyuan Huang
admin_assigned
2025-04-02T12:29:56.926Z
63c7a33121bd95f80ed74652
/avatars/7dd59afea785a2bff0ec2b757abd474e.svg
Siyuan Huang
user
thuhsy
ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via
  Residual Learning
Human hands play a central role in interacting, motivating increasing
research in dexterous robotic manipulation. Data-driven embodied AI algorithms
demand precise, large-scale, human-like manipulation sequences, which are
challenging to obtain with conventional reinforcement learning or real-world
teleoperation. To address this, we introduce ManipTrans, a novel two-stage
method for efficiently transferring human bimanual skills to dexterous robotic
hands in simulation. ManipTrans first pre-trains a generalist trajectory
imitator to mimic hand motion, then fine-tunes a specific residual module under
interaction constraints, enabling efficient learning and accurate execution of
complex bimanual tasks. Experiments show that ManipTrans surpasses
state-of-the-art methods in success rate, fidelity, and efficiency. Leveraging
ManipTrans, we transfer multiple hand-object datasets to robotic hands,
creating DexManipNet, a large-scale dataset featuring previously unexplored
tasks like pen capping and bottle unscrewing. DexManipNet comprises 3.3K
episodes of robotic manipulation and is easily extensible, facilitating further
policy training for dexterous hands and enabling real-world deployments.
ManipTrans is a two-stage method that efficiently transfers human bimanual skills to robotic hands in simulation, achieving high success rates and fidelity in complex tasks.
trajectory imitation residual module interaction constraints dexterous robotic hands bimanual tasks DexManipNet pen capping bottle unscrewing