title: Towards Synergistic, Generalized, and Efficient Dual-System for Robotic
    Manipulation
summary: The increasing demand for versatile robotic systems to operate in diverse and
  dynamic environments has emphasized the importance of a generalist policy,
  which leverages a large cross-embodiment data corpus to facilitate broad
  adaptability and high-level reasoning. However, the generalist would struggle
  with inefficient inference and cost-expensive training. The specialist policy,
  instead, is curated for specific domain data and excels at task-level precision
  with efficiency. Yet, it lacks the generalization capacity for a wide range of
  applications. Inspired by these observations, we introduce RoboDual, a
  synergistic dual-system that supplements the merits of both generalist and
  specialist policy. A diffusion transformer-based specialist is devised for
  multi-step action rollouts, exquisitely conditioned on the high-level task
  understanding and discretized action output of a vision-language-action (VLA)
  based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in
  real-world setting and 12% gain on CALVIN by introducing a specialist policy
  with merely 20M trainable parameters. It maintains strong performance with 5%
  of demonstration data only, and enables a 3.8 times higher control frequency in
  real-world deployment. Code would be made publicly available. Our project page
  is hosted at: https://opendrivelab.com/RoboDual/
paper_authors._id: 670f53a8d75f11435280d907
paper_authors.name: Qingwen Bu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-16T09:43:11.706Z
paper_authors._id: 64ac1f169dcc5787461468a4
paper_authors.avatarUrl: /avatars/c031a75989147009b7850df4eddfcb27.svg
paper_authors.fullname: Qingwen Bu
paper_authors.type: user
paper_authors.user: qwbu
paper_authors._id: 670f53a8d75f11435280d908
paper_authors.name: Hongyang Li
paper_authors._id: 670f53a8d75f11435280d909
paper_authors.name: Li Chen
paper_authors._id: 670f53a8d75f11435280d90a
paper_authors.name: Jisong Cai
paper_authors._id: 670f53a8d75f11435280d90b
paper_authors.name: Jia Zeng
paper_authors._id: 670f53a8d75f11435280d90c
paper_authors.name: Heming Cui
paper_authors._id: 670f53a8d75f11435280d90d
paper_authors.name: Maoqing Yao
paper_authors._id: 670f53a8d75f11435280d90e
paper_authors.name: Yu Qiao
paper_title: Towards Synergistic, Generalized, and Efficient Dual-System for Robotic
    Manipulation
paper_summary: The increasing demand for versatile robotic systems to operate in diverse and
  dynamic environments has emphasized the importance of a generalist policy,
  which leverages a large cross-embodiment data corpus to facilitate broad
  adaptability and high-level reasoning. However, the generalist would struggle
  with inefficient inference and cost-expensive training. The specialist policy,
  instead, is curated for specific domain data and excels at task-level precision
  with efficiency. Yet, it lacks the generalization capacity for a wide range of
  applications. Inspired by these observations, we introduce RoboDual, a
  synergistic dual-system that supplements the merits of both generalist and
  specialist policy. A diffusion transformer-based specialist is devised for
  multi-step action rollouts, exquisitely conditioned on the high-level task
  understanding and discretized action output of a vision-language-action (VLA)
  based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in
  real-world setting and 12% gain on CALVIN by introducing a specialist policy
  with merely 20M trainable parameters. It maintains strong performance with 5%
  of demonstration data only, and enables a 3.8 times higher control frequency in
  real-world deployment. Code would be made publicly available. Our project page
  is hosted at: https://opendrivelab.com/RoboDual/
paper_ai_summary: RoboDual integrates a diffusion transformer specialist with a vision-language-action generalist to enhance robotic adaptability and precision while maintaining efficiency.
paper_ai_keywords: diffusion transformer multi-step action rollouts vision-language-action high-level task understanding discretized action output OpenVLA CALVIN real-world setting control frequency