Towards Synergistic, Generalized, and Efficient Dual-System for Robotic
  Manipulation
The increasing demand for versatile robotic systems to operate in diverse and
dynamic environments has emphasized the importance of a generalist policy,
which leverages a large cross-embodiment data corpus to facilitate broad
adaptability and high-level reasoning. However, the generalist would struggle
with inefficient inference and cost-expensive training. The specialist policy,
instead, is curated for specific domain data and excels at task-level precision
with efficiency. Yet, it lacks the generalization capacity for a wide range of
applications. Inspired by these observations, we introduce RoboDual, a
synergistic dual-system that supplements the merits of both generalist and
specialist policy. A diffusion transformer-based specialist is devised for
multi-step action rollouts, exquisitely conditioned on the high-level task
understanding and discretized action output of a vision-language-action (VLA)
based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in
real-world setting and 12% gain on CALVIN by introducing a specialist policy
with merely 20M trainable parameters. It maintains strong performance with 5%
of demonstration data only, and enables a 3.8 times higher control frequency in
real-world deployment. Code would be made publicly available. Our project page
is hosted at: https://opendrivelab.com/RoboDual/
670f53a8d75f11435280d907
Qingwen Bu
claimed_verified
2024-10-16T09:43:11.706Z
64ac1f169dcc5787461468a4
/avatars/c031a75989147009b7850df4eddfcb27.svg
Qingwen Bu
user
qwbu
670f53a8d75f11435280d908
Hongyang Li
670f53a8d75f11435280d909
Li Chen
670f53a8d75f11435280d90a
Jisong Cai
670f53a8d75f11435280d90b
Jia Zeng
670f53a8d75f11435280d90c
Heming Cui
670f53a8d75f11435280d90d
Maoqing Yao
670f53a8d75f11435280d90e
Yu Qiao
Towards Synergistic, Generalized, and Efficient Dual-System for Robotic
  Manipulation
The increasing demand for versatile robotic systems to operate in diverse and
dynamic environments has emphasized the importance of a generalist policy,
which leverages a large cross-embodiment data corpus to facilitate broad
adaptability and high-level reasoning. However, the generalist would struggle
with inefficient inference and cost-expensive training. The specialist policy,
instead, is curated for specific domain data and excels at task-level precision
with efficiency. Yet, it lacks the generalization capacity for a wide range of
applications. Inspired by these observations, we introduce RoboDual, a
synergistic dual-system that supplements the merits of both generalist and
specialist policy. A diffusion transformer-based specialist is devised for
multi-step action rollouts, exquisitely conditioned on the high-level task
understanding and discretized action output of a vision-language-action (VLA)
based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in
real-world setting and 12% gain on CALVIN by introducing a specialist policy
with merely 20M trainable parameters. It maintains strong performance with 5%
of demonstration data only, and enables a 3.8 times higher control frequency in
real-world deployment. Code would be made publicly available. Our project page
is hosted at: https://opendrivelab.com/RoboDual/
RoboDual integrates a diffusion transformer specialist with a vision-language-action generalist to enhance robotic adaptability and precision while maintaining efficiency.
diffusion transformer multi-step action rollouts vision-language-action high-level task understanding discretized action output OpenVLA CALVIN real-world setting control frequency