title: Robotic Table Tennis: A Case Study into a High Speed Learning System
summary: We present a deep-dive into a real-world robotic learning system that, in
  previous work, was shown to be capable of hundreds of table tennis rallies with
  a human and has the ability to precisely return the ball to desired targets.
  This system puts together a highly optimized perception subsystem, a high-speed
  low-latency robot controller, a simulation paradigm that can prevent damage in
  the real world and also train policies for zero-shot transfer, and automated
  real world environment resets that enable autonomous training and evaluation on
  physical robots. We complement a complete system description, including
  numerous design decisions that are typically not widely disseminated, with a
  collection of studies that clarify the importance of mitigating various sources
  of latency, accounting for training and deployment distribution shifts,
  robustness of the perception system, sensitivity to policy hyper-parameters,
  and choice of action space. A video demonstrating the components of the system
  and details of experimental results can be found at
  https://youtu.be/uFcnWjB42I0.
paper_authors._id: 64fab6f467096272ae5d3cf9
paper_authors.name: David B. D'Ambrosio
paper_authors._id: 64fab6f467096272ae5d3cfa
paper_authors.name: Jonathan Abelian
paper_authors._id: 64fab6f467096272ae5d3cfb
paper_authors.name: Saminda Abeyruwan
paper_authors._id: 64fab6f467096272ae5d3cfc
paper_authors.name: Michael Ahn
paper_authors._id: 64fab6f467096272ae5d3cfd
paper_authors.name: Alex Bewley
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-08T11:43:27.318Z
paper_authors._id: 6492dfe783ff9ba7a9bbdeea
paper_authors.avatarUrl: /avatars/e1d8be8bb68200230cdf05497479e6f4.svg
paper_authors.fullname: Alex Bewley
paper_authors.type: user
paper_authors.user: bewley
paper_authors._id: 64fab6f467096272ae5d3cfe
paper_authors.name: Justin Boyd
paper_authors._id: 64fab6f467096272ae5d3cff
paper_authors.name: Krzysztof Choromanski
paper_authors._id: 64fab6f467096272ae5d3d00
paper_authors.name: Omar Cortes
paper_authors._id: 64fab6f467096272ae5d3d01
paper_authors.name: Erwin Coumans
paper_authors._id: 64fab6f467096272ae5d3d02
paper_authors.name: Tianli Ding
paper_authors._id: 64fab6f467096272ae5d3d03
paper_authors.name: Wenbo Gao
paper_authors._id: 64fab6f467096272ae5d3d04
paper_authors.name: Laura Graesser
paper_authors._id: 64fab6f467096272ae5d3d05
paper_authors.name: Atil Iscen
paper_authors._id: 64fab6f467096272ae5d3d06
paper_authors.name: Navdeep Jaitly
paper_authors._id: 64fab6f467096272ae5d3d07
paper_authors.name: Deepali Jain
paper_authors._id: 64fab6f467096272ae5d3d08
paper_authors.name: Juhana Kangaspunta
paper_authors._id: 64fab6f467096272ae5d3d09
paper_authors.name: Satoshi Kataoka
paper_authors._id: 64fab6f467096272ae5d3d0a
paper_authors.name: Gus Kouretas
paper_authors._id: 64fab6f467096272ae5d3d0b
paper_authors.name: Yuheng Kuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-08T13:05:03.687Z
paper_authors._id: 6312dd4f95407887cb7c9cd7
paper_authors.avatarUrl: /avatars/44a0e80ca9bbd2138eecc6b2a8b8b226.svg
paper_authors.fullname: Yuheng Kuang
paper_authors.type: user
paper_authors.user: kuangyh
paper_authors._id: 64fab6f467096272ae5d3d0c
paper_authors.name: Nevena Lazic
paper_authors._id: 64fab6f467096272ae5d3d0d
paper_authors.name: Corey Lynch
paper_authors._id: 64fab6f467096272ae5d3d0e
paper_authors.name: Reza Mahjourian
paper_authors._id: 64fab6f467096272ae5d3d0f
paper_authors.name: Sherry Q. Moore
paper_authors._id: 64fab6f467096272ae5d3d10
paper_authors.name: Thinh Nguyen
paper_authors._id: 64fab6f467096272ae5d3d11
paper_authors.name: Ken Oslund
paper_authors._id: 64fab6f467096272ae5d3d12
paper_authors.name: Barney J Reed
paper_authors._id: 64fab6f467096272ae5d3d13
paper_authors.name: Krista Reymann
paper_authors._id: 64fab6f467096272ae5d3d14
paper_authors.name: Pannag R. Sanketi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-09T07:49:15.902Z
paper_authors._id: 66b56f73fc66a466e6104bfb
paper_authors.avatarUrl: /avatars/74855aaf580144fafead4b93d6a24a46.svg
paper_authors.fullname: Pannag Sanketi
paper_authors.type: user
paper_authors.user: prsanky
paper_authors._id: 64fab6f467096272ae5d3d15
paper_authors.name: Anish Shankar
paper_authors._id: 64fab6f467096272ae5d3d16
paper_authors.name: Pierre Sermanet
paper_authors._id: 64fab6f467096272ae5d3d17
paper_authors.name: Vikas Sindhwani
paper_authors._id: 64fab6f467096272ae5d3d18
paper_authors.name: Avi Singh
paper_authors._id: 64fab6f467096272ae5d3d19
paper_authors.name: Vincent Vanhoucke
paper_authors._id: 64fab6f467096272ae5d3d1a
paper_authors.name: Grace Vesom
paper_authors._id: 64fab6f467096272ae5d3d1b
paper_authors.name: Peng Xu
paper_title: Robotic Table Tennis: A Case Study into a High Speed Learning System
paper_summary: We present a deep-dive into a real-world robotic learning system that, in
  previous work, was shown to be capable of hundreds of table tennis rallies with
  a human and has the ability to precisely return the ball to desired targets.
  This system puts together a highly optimized perception subsystem, a high-speed
  low-latency robot controller, a simulation paradigm that can prevent damage in
  the real world and also train policies for zero-shot transfer, and automated
  real world environment resets that enable autonomous training and evaluation on
  physical robots. We complement a complete system description, including
  numerous design decisions that are typically not widely disseminated, with a
  collection of studies that clarify the importance of mitigating various sources
  of latency, accounting for training and deployment distribution shifts,
  robustness of the perception system, sensitivity to policy hyper-parameters,
  and choice of action space. A video demonstrating the components of the system
  and details of experimental results can be found at
  https://youtu.be/uFcnWjB42I0.
paper_ai_keywords: robotic learning system table tennis rallies perception subsystem high-speed low-latency robot controller simulation paradigm zero-shot transfer automated real world environment resets autonomous training and evaluation