title: FAST: Efficient Action Tokenization for Vision-Language-Action Models
summary: Autoregressive sequence models, such as Transformer-based vision-language
  action (VLA) policies, can be tremendously effective for capturing complex and
  generalizable robotic behaviors. However, such models require us to choose a
  tokenization of our continuous action signals, which determines how the
  discrete symbols predicted by the model map to continuous robot actions. We
  find that current approaches for robot action tokenization, based on simple
  per-dimension, per-timestep binning schemes, typically perform poorly when
  learning dexterous skills from high-frequency robot data. To address this
  challenge, we propose a new compression-based tokenization scheme for robot
  actions, based on the discrete cosine transform. Our tokenization approach,
  Frequency-space Action Sequence Tokenization (FAST), enables us to train
  autoregressive VLAs for highly dexterous and high-frequency tasks where
  standard discretization methods fail completely. Based on FAST, we release
  FAST+, a universal robot action tokenizer, trained on 1M real robot action
  trajectories. It can be used as a black-box tokenizer for a wide range of robot
  action sequences, with diverse action spaces and control frequencies. Finally,
  we show that, when combined with the pi0 VLA, our method can scale to training
  on 10k hours of robot data and match the performance of diffusion VLAs, while
  reducing training time by up to 5x.
paper_authors._id: 6789e918e1b3fda757de947f
paper_authors.name: Karl Pertsch
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:36:46.947Z
paper_authors._id: 65d4c1ff29b4ac81c265e6e6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65d4c1ff29b4ac81c265e6e6/GXgs28okxGfpBdqhxov9-.png
paper_authors.fullname: Karl Pertsch
paper_authors.type: user
paper_authors.user: KarlP
paper_authors._id: 6789e918e1b3fda757de9480
paper_authors.name: Kyle Stachowicz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:36:59.830Z
paper_authors._id: 6307eabda670ed10f9d2571f
paper_authors.avatarUrl: /avatars/b0811b25ed4fb7e48dd380898049c764.svg
paper_authors.fullname: Kyle Stachowicz
paper_authors.type: user
paper_authors.user: kylestach
paper_authors._id: 6789e918e1b3fda757de9481
paper_authors.name: Brian Ichter
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:37:05.229Z
paper_authors._id: 633a26ab474cfeb1a864dc56
paper_authors.avatarUrl: /avatars/cd40accd894fc78810fc0d5108f413e9.svg
paper_authors.fullname: Brian I
paper_authors.type: user
paper_authors.user: brianichter
paper_authors._id: 6789e918e1b3fda757de9482
paper_authors.name: Danny Driess
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:37:10.829Z
paper_authors._id: 67225875b46c703941fa7967
paper_authors.avatarUrl: /avatars/7c89fbdd9a135210209bcd0cbfe7988a.svg
paper_authors.fullname: Danny Driess
paper_authors.type: user
paper_authors.user: dannydriess
paper_authors._id: 6789e918e1b3fda757de9483
paper_authors.name: Suraj Nair
paper_authors._id: 6789e918e1b3fda757de9484
paper_authors.name: Quan Vuong
paper_authors._id: 6789e918e1b3fda757de9485
paper_authors.name: Oier Mees
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:37:30.435Z
paper_authors._id: 663a7190d9dee283e3f56150
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/663a7190d9dee283e3f56150/zoYBdFIQPGWTS0R7bpg07.jpeg
paper_authors.fullname: Oier Mees
paper_authors.type: user
paper_authors.user: oier-mees
paper_authors._id: 6789e918e1b3fda757de9486
paper_authors.name: Chelsea Finn
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:37:36.079Z
paper_authors._id: 64ac22a9193f0a807deb673d
paper_authors.avatarUrl: /avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
paper_authors.fullname: Chelsea Finn
paper_authors.type: user
paper_authors.user: cbfinn
paper_authors._id: 6789e918e1b3fda757de9487
paper_authors.name: Sergey Levine
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:37:41.213Z
paper_authors._id: 665ce54120a307a3754849dd
paper_authors.avatarUrl: /avatars/e698726e9be61dd50ce2efe372ed5dac.svg
paper_authors.fullname: Sergey Levine
paper_authors.type: user
paper_authors.user: svlevine
paper_title: FAST: Efficient Action Tokenization for Vision-Language-Action Models
paper_summary: Autoregressive sequence models, such as Transformer-based vision-language
  action (VLA) policies, can be tremendously effective for capturing complex and
  generalizable robotic behaviors. However, such models require us to choose a
  tokenization of our continuous action signals, which determines how the
  discrete symbols predicted by the model map to continuous robot actions. We
  find that current approaches for robot action tokenization, based on simple
  per-dimension, per-timestep binning schemes, typically perform poorly when
  learning dexterous skills from high-frequency robot data. To address this
  challenge, we propose a new compression-based tokenization scheme for robot
  actions, based on the discrete cosine transform. Our tokenization approach,
  Frequency-space Action Sequence Tokenization (FAST), enables us to train
  autoregressive VLAs for highly dexterous and high-frequency tasks where
  standard discretization methods fail completely. Based on FAST, we release
  FAST+, a universal robot action tokenizer, trained on 1M real robot action
  trajectories. It can be used as a black-box tokenizer for a wide range of robot
  action sequences, with diverse action spaces and control frequencies. Finally,
  we show that, when combined with the pi0 VLA, our method can scale to training
  on 10k hours of robot data and match the performance of diffusion VLAs, while
  reducing training time by up to 5x.
paper_ai_keywords: autoregressive sequence models Transformer-based vision-language action (VLA) policies tokenization continuous action signals discrete symbols continuous robot actions per-dimension, per-timestep binning schemes discrete cosine transform Frequency-space Action Sequence Tokenization (FAST) dexterous skills high-frequency robot data standard discretization methods universal robot action tokenizer pi0 VLA diffusion VLAs