title: Universal Manipulation Interface: In-The-Wild Robot Teaching Without
    In-The-Wild Robots
summary: We present Universal Manipulation Interface (UMI) -- a data collection and
  policy learning framework that allows direct skill transfer from in-the-wild
  human demonstrations to deployable robot policies. UMI employs hand-held
  grippers coupled with careful interface design to enable portable, low-cost,
  and information-rich data collection for challenging bimanual and dynamic
  manipulation demonstrations. To facilitate deployable policy learning, UMI
  incorporates a carefully designed policy interface with inference-time latency
  matching and a relative-trajectory action representation. The resulting learned
  policies are hardware-agnostic and deployable across multiple robot platforms.
  Equipped with these features, UMI framework unlocks new robot manipulation
  capabilities, allowing zero-shot generalizable dynamic, bimanual, precise, and
  long-horizon behaviors, by only changing the training data for each task. We
  demonstrate UMI's versatility and efficacy with comprehensive real-world
  experiments, where policies learned via UMI zero-shot generalize to novel
  environments and objects when trained on diverse human demonstrations. UMI's
  hardware and software system is open-sourced at https://umi-gripper.github.io.
paper_authors._id: 65d2edfab3eb7eed71f9e812
paper_authors.name: Cheng Chi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-20T08:39:01.601Z
paper_authors._id: 630440995c70c21d0eaea552
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630440995c70c21d0eaea552/FxrpcUgnnnGWV_9k-KcBa.jpeg
paper_authors.fullname: Cheng Chi
paper_authors.type: user
paper_authors.user: chengchi
paper_authors._id: 65d2edfab3eb7eed71f9e813
paper_authors.name: Zhenjia Xu
paper_authors._id: 65d2edfab3eb7eed71f9e814
paper_authors.name: Chuer Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T15:17:31.863Z
paper_authors._id: 63efc1b3fc92a63ac8116ff2
paper_authors.avatarUrl: /avatars/60af94291eba0a6b5b43c5c1b58fbe70.svg
paper_authors.fullname: Pan
paper_authors.type: user
paper_authors.user: Chuer
paper_authors._id: 65d2edfab3eb7eed71f9e815
paper_authors.name: Eric Cousineau
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-19T15:17:56.614Z
paper_authors._id: 630a4b191d1650ec2f8e81a1
paper_authors.avatarUrl: /avatars/0335fe1020963add3d25f831c5b6b047.svg
paper_authors.fullname: Eric Cousineau
paper_authors.type: user
paper_authors.user: EricCousineau-TRI
paper_authors._id: 65d2edfab3eb7eed71f9e816
paper_authors.name: Benjamin Burchfiel
paper_authors._id: 65d2edfab3eb7eed71f9e817
paper_authors.name: Siyuan Feng
paper_authors._id: 65d2edfab3eb7eed71f9e818
paper_authors.name: Russ Tedrake
paper_authors._id: 65d2edfab3eb7eed71f9e819
paper_authors.name: Shuran Song
paper_title: Universal Manipulation Interface: In-The-Wild Robot Teaching Without
    In-The-Wild Robots
paper_summary: We present Universal Manipulation Interface (UMI) -- a data collection and
  policy learning framework that allows direct skill transfer from in-the-wild
  human demonstrations to deployable robot policies. UMI employs hand-held
  grippers coupled with careful interface design to enable portable, low-cost,
  and information-rich data collection for challenging bimanual and dynamic
  manipulation demonstrations. To facilitate deployable policy learning, UMI
  incorporates a carefully designed policy interface with inference-time latency
  matching and a relative-trajectory action representation. The resulting learned
  policies are hardware-agnostic and deployable across multiple robot platforms.
  Equipped with these features, UMI framework unlocks new robot manipulation
  capabilities, allowing zero-shot generalizable dynamic, bimanual, precise, and
  long-horizon behaviors, by only changing the training data for each task. We
  demonstrate UMI's versatility and efficacy with comprehensive real-world
  experiments, where policies learned via UMI zero-shot generalize to novel
  environments and objects when trained on diverse human demonstrations. UMI's
  hardware and software system is open-sourced at https://umi-gripper.github.io.