title: AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation
    System
summary: Vision-based teleoperation offers the possibility to endow robots with
  human-level intelligence to physically interact with the environment, while
  only requiring low-cost camera sensors. However, current vision-based
  teleoperation systems are designed and engineered towards a particular robot
  model and deploy environment, which scales poorly as the pool of the robot
  models expands and the variety of the operating environment increases. In this
  paper, we propose AnyTeleop, a unified and general teleoperation system to
  support multiple different arms, hands, realities, and camera configurations
  within a single system. Although being designed to provide great flexibility to
  the choice of simulators and real hardware, our system can still achieve great
  performance. For real-world experiments, AnyTeleop can outperform a previous
  system that was designed for a specific robot hardware with a higher success
  rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to
  better imitation learning performance, compared with a previous system that is
  particularly designed for that simulator. Project page: http://anyteleop.com/.
paper_authors._id: 64ace52bc4e03b989f7fe5d1
paper_authors.name: Yuzhe Qin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-12T07:29:06.672Z
paper_authors._id: 64ad64c2c1da7c4dbc7b19c5
paper_authors.avatarUrl: /avatars/98efbab6c65405e8701c64c290a7a87c.svg
paper_authors.fullname: Yuzhe Qin
paper_authors.type: user
paper_authors.user: yuzheqin
paper_authors._id: 64ace52bc4e03b989f7fe5d2
paper_authors.name: Wei Yang
paper_authors._id: 64ace52bc4e03b989f7fe5d3
paper_authors.name: Binghao Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-13T08:32:55.325Z
paper_authors._id: 64af4da473151ec37ee461a5
paper_authors.avatarUrl: /avatars/fea36cd0fa995c9827edce40286b9245.svg
paper_authors.fullname: Binghao Huang
paper_authors.type: user
paper_authors.user: Binghaohuang
paper_authors._id: 64ace52bc4e03b989f7fe5d4
paper_authors.name: Karl Van Wyk
paper_authors._id: 64ace52bc4e03b989f7fe5d5
paper_authors.name: Hao Su
paper_authors._id: 64ace52bc4e03b989f7fe5d6
paper_authors.name: Xiaolong Wang
paper_authors._id: 64ace52bc4e03b989f7fe5d7
paper_authors.name: Yu-Wei Chao
paper_authors._id: 64ace52bc4e03b989f7fe5d8
paper_authors.name: Dietor Fox
paper_title: AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation
    System
paper_summary: Vision-based teleoperation offers the possibility to endow robots with
  human-level intelligence to physically interact with the environment, while
  only requiring low-cost camera sensors. However, current vision-based
  teleoperation systems are designed and engineered towards a particular robot
  model and deploy environment, which scales poorly as the pool of the robot
  models expands and the variety of the operating environment increases. In this
  paper, we propose AnyTeleop, a unified and general teleoperation system to
  support multiple different arms, hands, realities, and camera configurations
  within a single system. Although being designed to provide great flexibility to
  the choice of simulators and real hardware, our system can still achieve great
  performance. For real-world experiments, AnyTeleop can outperform a previous
  system that was designed for a specific robot hardware with a higher success
  rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to
  better imitation learning performance, compared with a previous system that is
  particularly designed for that simulator. Project page: http://anyteleop.com/.
paper_ai_summary: AnyTeleop is a flexible and unified teleoperation system that supports various robotic and environmental configurations, achieving superior performance compared to specialized systems in both real-world and simulated environments.
paper_ai_keywords: vision-based teleoperation human-level intelligence low-cost camera sensors unified teleoperation system multiple robot models flexible simulators real hardware real-world experiments imitation learning simulation