RVT-2: Learning Precise Manipulation from Few Demonstrations
In this work, we study how to build a robotic system that can solve multiple
3D manipulation tasks given language instructions. To be useful in industrial
and household domains, such a system should be capable of learning new tasks
with few demonstrations and solving them precisely. Prior works, like PerAct
and RVT, have studied this problem, however, they often struggle with tasks
requiring high precision. We study how to make them more effective, precise,
and fast. Using a combination of architectural and system-level improvements,
we propose RVT-2, a multitask 3D manipulation model that is 6X faster in
training and 2X faster in inference than its predecessor RVT. RVT-2 achieves a
new state-of-the-art on RLBench, improving the success rate from 65% to 82%.
RVT-2 is also effective in the real world, where it can learn tasks requiring
high precision, like picking up and inserting plugs, with just 10
demonstrations. Visual results, code, and trained model are provided at:
https://robotic-view-transformer-2.github.io/.
66709b08e12fa6cc5f3012aa
Ankit Goyal
claimed_verified
2024-07-09T07:42:24.238Z
666c6f67bc840e6748554a91
/avatars/72121715e14f720e5c1d029b7f00d55d.svg
Ankit Goyal
user
ankgoyal
66709b08e12fa6cc5f3012ab
Valts Blukis
66709b08e12fa6cc5f3012ac
Jie Xu
claimed_verified
2024-08-16T21:02:50.480Z
66bf785fdccd1164803e0270
/avatars/e762628f8b8f18cefd6a8ed3a7810ec9.svg
Jie Xu
user
xujie7979
66709b08e12fa6cc5f3012ad
Yijie Guo
66709b08e12fa6cc5f3012ae
Yu-Wei Chao
66709b08e12fa6cc5f3012af
Dieter Fox
RVT-2: Learning Precise Manipulation from Few Demonstrations
In this work, we study how to build a robotic system that can solve multiple
3D manipulation tasks given language instructions. To be useful in industrial
and household domains, such a system should be capable of learning new tasks
with few demonstrations and solving them precisely. Prior works, like PerAct
and RVT, have studied this problem, however, they often struggle with tasks
requiring high precision. We study how to make them more effective, precise,
and fast. Using a combination of architectural and system-level improvements,
we propose RVT-2, a multitask 3D manipulation model that is 6X faster in
training and 2X faster in inference than its predecessor RVT. RVT-2 achieves a
new state-of-the-art on RLBench, improving the success rate from 65% to 82%.
RVT-2 is also effective in the real world, where it can learn tasks requiring
high precision, like picking up and inserting plugs, with just 10
demonstrations. Visual results, code, and trained model are provided at:
https://robotic-view-transformer-2.github.io/.
A multitask 3D manipulation model, RVT-2, achieves high precision and speed in task learning and execution through architectural and system-level optimizations.
PerAct RVT RVT-2 3D manipulation language instructions multitask model RLBench precision demonstrations training inference