title: CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive
    Prediction
summary: In robotic visuomotor policy learning, diffusion-based models have achieved
  significant success in improving the accuracy of action trajectory generation
  compared to traditional autoregressive models. However, they suffer from
  inefficiency due to multiple denoising steps and limited flexibility from
  complex constraints. In this paper, we introduce Coarse-to-Fine AutoRegressive
  Policy (CARP), a novel paradigm for visuomotor policy learning that redefines
  the autoregressive action generation process as a coarse-to-fine, next-scale
  approach. CARP decouples action generation into two stages: first, an action
  autoencoder learns multi-scale representations of the entire action sequence;
  then, a GPT-style transformer refines the sequence prediction through a
  coarse-to-fine autoregressive process. This straightforward and intuitive
  approach produces highly accurate and smooth actions, matching or even
  surpassing the performance of diffusion-based policies while maintaining
  efficiency on par with autoregressive policies. We conduct extensive
  evaluations across diverse settings, including single-task and multi-task
  scenarios on state-based and image-based simulation benchmarks, as well as
  real-world tasks. CARP achieves competitive success rates, with up to a 10%
  improvement, and delivers 10x faster inference compared to state-of-the-art
  policies, establishing a high-performance, efficient, and flexible paradigm for
  action generation in robotic tasks.
paper_authors._id: 6757e0d412d7d9e5dcffaf95
paper_authors.name: Zhefei Gong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T13:08:37.754Z
paper_authors._id: 64cc96cb275c76304617ba4e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64cc96cb275c76304617ba4e/6-z5s0mBrbu3TYcebXou5.png
paper_authors.fullname: Jeffrey
paper_authors.type: user
paper_authors.user: zhefeigong
paper_authors._id: 6757e0d412d7d9e5dcffaf96
paper_authors.name: Pengxiang Ding
paper_authors._id: 6757e0d412d7d9e5dcffaf97
paper_authors.name: Shangke Lyu
paper_authors._id: 6757e0d412d7d9e5dcffaf98
paper_authors.name: Siteng Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T13:09:32.865Z
paper_authors._id: 65fd82762bf2cd20ddaa193f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/yBYbWp_mT7UusYdkqtAvw.png
paper_authors.fullname: Siteng Huang
paper_authors.type: user
paper_authors.user: huangsiteng
paper_authors._id: 6757e0d412d7d9e5dcffaf99
paper_authors.name: Mingyang Sun
paper_authors._id: 6757e0d412d7d9e5dcffaf9a
paper_authors.name: Wei Zhao
paper_authors._id: 6757e0d412d7d9e5dcffaf9b
paper_authors.name: Zhaoxin Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-10T13:10:01.836Z
paper_authors._id: 6462eab02538819c72a0b2cc
paper_authors.avatarUrl: /avatars/b4ece21ce35da7506db9d8bef4abcf68.svg
paper_authors.fullname: Zhaoxin Fan
paper_authors.type: user
paper_authors.user: Jason007x
paper_authors._id: 6757e0d412d7d9e5dcffaf9c
paper_authors.name: Donglin Wang
paper_title: CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive
    Prediction
paper_summary: In robotic visuomotor policy learning, diffusion-based models have achieved
  significant success in improving the accuracy of action trajectory generation
  compared to traditional autoregressive models. However, they suffer from
  inefficiency due to multiple denoising steps and limited flexibility from
  complex constraints. In this paper, we introduce Coarse-to-Fine AutoRegressive
  Policy (CARP), a novel paradigm for visuomotor policy learning that redefines
  the autoregressive action generation process as a coarse-to-fine, next-scale
  approach. CARP decouples action generation into two stages: first, an action
  autoencoder learns multi-scale representations of the entire action sequence;
  then, a GPT-style transformer refines the sequence prediction through a
  coarse-to-fine autoregressive process. This straightforward and intuitive
  approach produces highly accurate and smooth actions, matching or even
  surpassing the performance of diffusion-based policies while maintaining
  efficiency on par with autoregressive policies. We conduct extensive
  evaluations across diverse settings, including single-task and multi-task
  scenarios on state-based and image-based simulation benchmarks, as well as
  real-world tasks. CARP achieves competitive success rates, with up to a 10%
  improvement, and delivers 10x faster inference compared to state-of-the-art
  policies, establishing a high-performance, efficient, and flexible paradigm for
  action generation in robotic tasks.
paper_ai_summary: CARP, a coarse-to-fine autoregressive policy, improves action generation efficiency and accuracy in robotic tasks by using an action autoencoder and GPT-style transformer, outperforming diffusion-based and traditional policies.
paper_ai_keywords: diffusion-based models autoregressive models action trajectory generation action autoencoder GPT-style transformer coarse-to-fine multi-scale representations single-task multi-task state-based image-based simulation benchmarks real-world tasks