title: DexUMI: Using Human Hand as the Universal Manipulation Interface for
    Dexterous Manipulation
summary: We present DexUMI - a data collection and policy learning framework that uses
  the human hand as the natural interface to transfer dexterous manipulation
  skills to various robot hands. DexUMI includes hardware and software
  adaptations to minimize the embodiment gap between the human hand and various
  robot hands. The hardware adaptation bridges the kinematics gap using a
  wearable hand exoskeleton. It allows direct haptic feedback in manipulation
  data collection and adapts human motion to feasible robot hand motion. The
  software adaptation bridges the visual gap by replacing the human hand in video
  data with high-fidelity robot hand inpainting. We demonstrate DexUMI's
  capabilities through comprehensive real-world experiments on two different
  dexterous robot hand hardware platforms, achieving an average task success rate
  of 86%.
paper_authors._id: 683b8af5091615f46fabadde
paper_authors.name: Mengda Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:42:10.223Z
paper_authors._id: 655a50a850b9a14799165d53
paper_authors.avatarUrl: /avatars/6bb37ac0b6840771ff7a6a6da4192ace.svg
paper_authors.fullname: Mengda Xu
paper_authors.type: user
paper_authors.user: mengdaxu
paper_authors._id: 683b8af5091615f46fabaddf
paper_authors.name: Han Zhang
paper_authors._id: 683b8af5091615f46fabade0
paper_authors.name: Yifan Hou
paper_authors._id: 683b8af5091615f46fabade1
paper_authors.name: Zhenjia Xu
paper_authors._id: 683b8af5091615f46fabade2
paper_authors.name: Linxi Fan
paper_authors._id: 683b8af5091615f46fabade3
paper_authors.name: Manuela Veloso
paper_authors._id: 683b8af5091615f46fabade4
paper_authors.name: Shuran Song
paper_title: DexUMI: Using Human Hand as the Universal Manipulation Interface for
    Dexterous Manipulation
paper_summary: We present DexUMI - a data collection and policy learning framework that uses
  the human hand as the natural interface to transfer dexterous manipulation
  skills to various robot hands. DexUMI includes hardware and software
  adaptations to minimize the embodiment gap between the human hand and various
  robot hands. The hardware adaptation bridges the kinematics gap using a
  wearable hand exoskeleton. It allows direct haptic feedback in manipulation
  data collection and adapts human motion to feasible robot hand motion. The
  software adaptation bridges the visual gap by replacing the human hand in video
  data with high-fidelity robot hand inpainting. We demonstrate DexUMI's
  capabilities through comprehensive real-world experiments on two different
  dexterous robot hand hardware platforms, achieving an average task success rate
  of 86%.
paper_ai_summary: DexUMI framework utilizes a wearable hand exoskeleton and high-fidelity robot hand inpainting to transfer dexterous manipulation skills from human hands to robot hands, achieving high task success rates.
paper_ai_keywords: wearable hand exoskeleton haptic feedback robot hand inpainting dexterous manipulation kinematics visual gap