title: Agile Catching with Whole-Body MPC and Blackbox Policy Learning
summary: We address a benchmark task in agile robotics: catching objects thrown at
  high-speed. This is a challenging task that involves tracking, intercepting,
  and cradling a thrown object with access only to visual observations of the
  object and the proprioceptive state of the robot, all within a fraction of a
  second. We present the relative merits of two fundamentally different solution
  strategies: (i) Model Predictive Control using accelerated constrained
  trajectory optimization, and (ii) Reinforcement Learning using zeroth-order
  optimization. We provide insights into various performance trade-offs including
  sample efficiency, sim-to-real transfer, robustness to distribution shifts, and
  whole-body multimodality via extensive on-hardware experiments. We conclude
  with proposals on fusing "classical" and "learning-based" techniques for agile
  robot control. Videos of our experiments may be found at
  https://sites.google.com/view/agile-catching
paper_authors._id: 648c028817c7ceb9b41a40a1
paper_authors.name: Saminda Abeyruwan
paper_authors._id: 648c028817c7ceb9b41a40a2
paper_authors.name: Alex Bewley
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-21T20:46:05.285Z
paper_authors._id: 6492dfe783ff9ba7a9bbdeea
paper_authors.avatarUrl: /avatars/e1d8be8bb68200230cdf05497479e6f4.svg
paper_authors.fullname: Alex Bewley
paper_authors.type: user
paper_authors.user: bewley
paper_authors._id: 648c028817c7ceb9b41a40a3
paper_authors.name: Nicholas M. Boffi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-23T17:09:35.679Z
paper_authors._id: 65afd8b7bd418aab3a677782
paper_authors.avatarUrl: /avatars/11aa33b32ef4b6aa27ab8e6c068afa14.svg
paper_authors.fullname: Nicholas Boffi
paper_authors.type: user
paper_authors.user: nmboffi
paper_authors._id: 648c028817c7ceb9b41a40a4
paper_authors.name: Krzysztof Choromanski
paper_authors._id: 648c028817c7ceb9b41a40a5
paper_authors.name: David D'Ambrosio
paper_authors._id: 648c028817c7ceb9b41a40a6
paper_authors.name: Deepali Jain
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-16T17:38:19.893Z
paper_authors._id: 648c741b39d2584ee46ebc7b
paper_authors.avatarUrl: /avatars/ad3e078c6df8b319ef5fa85e556f8a3c.svg
paper_authors.fullname: Deepali Jain
paper_authors.type: user
paper_authors.user: jaindeepali
paper_authors._id: 648c028817c7ceb9b41a40a7
paper_authors.name: Pannag Sanketi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-09T07:49:26.766Z
paper_authors._id: 66b56f73fc66a466e6104bfb
paper_authors.avatarUrl: /avatars/74855aaf580144fafead4b93d6a24a46.svg
paper_authors.fullname: Pannag Sanketi
paper_authors.type: user
paper_authors.user: prsanky
paper_authors._id: 648c028817c7ceb9b41a40a8
paper_authors.name: Anish Shankar
paper_authors._id: 648c028817c7ceb9b41a40a9
paper_authors.name: Vikas Sindhwani
paper_authors._id: 648c028817c7ceb9b41a40aa
paper_authors.name: Sumeet Singh
paper_authors._id: 648c028817c7ceb9b41a40ab
paper_authors.name: Jean-Jacques Slotine
paper_authors._id: 648c028817c7ceb9b41a40ac
paper_authors.name: Stephen Tu
paper_title: Agile Catching with Whole-Body MPC and Blackbox Policy Learning
paper_summary: We address a benchmark task in agile robotics: catching objects thrown at
  high-speed. This is a challenging task that involves tracking, intercepting,
  and cradling a thrown object with access only to visual observations of the
  object and the proprioceptive state of the robot, all within a fraction of a
  second. We present the relative merits of two fundamentally different solution
  strategies: (i) Model Predictive Control using accelerated constrained
  trajectory optimization, and (ii) Reinforcement Learning using zeroth-order
  optimization. We provide insights into various performance trade-offs including
  sample efficiency, sim-to-real transfer, robustness to distribution shifts, and
  whole-body multimodality via extensive on-hardware experiments. We conclude
  with proposals on fusing "classical" and "learning-based" techniques for agile
  robot control. Videos of our experiments may be found at
  https://sites.google.com/view/agile-catching