title: RVT-2: Learning Precise Manipulation from Few Demonstrations
summary: In this work, we study how to build a robotic system that can solve multiple
  3D manipulation tasks given language instructions. To be useful in industrial
  and household domains, such a system should be capable of learning new tasks
  with few demonstrations and solving them precisely. Prior works, like PerAct
  and RVT, have studied this problem, however, they often struggle with tasks
  requiring high precision. We study how to make them more effective, precise,
  and fast. Using a combination of architectural and system-level improvements,
  we propose RVT-2, a multitask 3D manipulation model that is 6X faster in
  training and 2X faster in inference than its predecessor RVT. RVT-2 achieves a
  new state-of-the-art on RLBench, improving the success rate from 65% to 82%.
  RVT-2 is also effective in the real world, where it can learn tasks requiring
  high precision, like picking up and inserting plugs, with just 10
  demonstrations. Visual results, code, and trained model are provided at:
  https://robotic-view-transformer-2.github.io/.
paper_authors._id: 66709b08e12fa6cc5f3012aa
paper_authors.name: Ankit Goyal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-09T07:42:24.238Z
paper_authors._id: 666c6f67bc840e6748554a91
paper_authors.avatarUrl: /avatars/72121715e14f720e5c1d029b7f00d55d.svg
paper_authors.fullname: Ankit Goyal
paper_authors.type: user
paper_authors.user: ankgoyal
paper_authors._id: 66709b08e12fa6cc5f3012ab
paper_authors.name: Valts Blukis
paper_authors._id: 66709b08e12fa6cc5f3012ac
paper_authors.name: Jie Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-16T21:02:50.480Z
paper_authors._id: 66bf785fdccd1164803e0270
paper_authors.avatarUrl: /avatars/e762628f8b8f18cefd6a8ed3a7810ec9.svg
paper_authors.fullname: Jie Xu
paper_authors.type: user
paper_authors.user: xujie7979
paper_authors._id: 66709b08e12fa6cc5f3012ad
paper_authors.name: Yijie Guo
paper_authors._id: 66709b08e12fa6cc5f3012ae
paper_authors.name: Yu-Wei Chao
paper_authors._id: 66709b08e12fa6cc5f3012af
paper_authors.name: Dieter Fox
paper_title: RVT-2: Learning Precise Manipulation from Few Demonstrations
paper_summary: In this work, we study how to build a robotic system that can solve multiple
  3D manipulation tasks given language instructions. To be useful in industrial
  and household domains, such a system should be capable of learning new tasks
  with few demonstrations and solving them precisely. Prior works, like PerAct
  and RVT, have studied this problem, however, they often struggle with tasks
  requiring high precision. We study how to make them more effective, precise,
  and fast. Using a combination of architectural and system-level improvements,
  we propose RVT-2, a multitask 3D manipulation model that is 6X faster in
  training and 2X faster in inference than its predecessor RVT. RVT-2 achieves a
  new state-of-the-art on RLBench, improving the success rate from 65% to 82%.
  RVT-2 is also effective in the real world, where it can learn tasks requiring
  high precision, like picking up and inserting plugs, with just 10
  demonstrations. Visual results, code, and trained model are provided at:
  https://robotic-view-transformer-2.github.io/.
paper_ai_summary: A multitask 3D manipulation model, RVT-2, achieves high precision and speed in task learning and execution through architectural and system-level optimizations.
paper_ai_keywords: PerAct RVT RVT-2 3D manipulation language instructions multitask model RLBench precision demonstrations training inference