title: Dita: Scaling Diffusion Transformer for Generalist
    Vision-Language-Action Policy
summary: While recent vision-language-action models trained on diverse robot datasets
  exhibit promising generalization capabilities with limited in-domain data,
  their reliance on compact action heads to predict discretized or continuous
  actions constrains adaptability to heterogeneous action spaces. We present
  Dita, a scalable framework that leverages Transformer architectures to directly
  denoise continuous action sequences through a unified multimodal diffusion
  process. Departing from prior methods that condition denoising on fused
  embeddings via shallow networks, Dita employs in-context conditioning --
  enabling fine-grained alignment between denoised actions and raw visual tokens
  from historical observations. This design explicitly models action deltas and
  environmental nuances. By scaling the diffusion action denoiser alongside the
  Transformer's scalability, Dita effectively integrates cross-embodiment
  datasets across diverse camera perspectives, observation scenes, tasks, and
  action spaces. Such synergy enhances robustness against various variances and
  facilitates the successful execution of long-horizon tasks. Evaluations across
  extensive benchmarks demonstrate state-of-the-art or comparative performance in
  simulation. Notably, Dita achieves robust real-world adaptation to
  environmental variances and complex long-horizon tasks through 10-shot
  finetuning, using only third-person camera inputs. The architecture establishes
  a versatile, lightweight and open-source baseline for generalist robot policy
  learning. Project Page: https://robodita.github.io.
paper_authors._id: 67e3e1e20706b07bfb2713d6
paper_authors.name: Zhi Hou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:43:49.995Z
paper_authors._id: 643fa1c318afbc4d1f3e5e59
paper_authors.avatarUrl: /avatars/f8f35355902b4cda72e9c6d768322fae.svg
paper_authors.fullname: Zhi Hou
paper_authors.type: user
paper_authors.user: zhihou
paper_authors._id: 67e3e1e20706b07bfb2713d7
paper_authors.name: Tianyi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:44:21.118Z
paper_authors._id: 64c9e86a6a26cddbecd9bae2
paper_authors.avatarUrl: /avatars/61a84989dbbc1898ebcba3236dbed039.svg
paper_authors.fullname: Tianyi Zhang
paper_authors.type: user
paper_authors.user: TianyiZhang0213
paper_authors._id: 67e3e1e20706b07bfb2713d8
paper_authors.name: Yuwen Xiong
paper_authors._id: 67e3e1e20706b07bfb2713d9
paper_authors.name: Haonan Duan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-27T09:04:53.498Z
paper_authors._id: 66ab30dfd456f0408b93f27b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66ab30dfd456f0408b93f27b/nps4Kni_eOExO5Z92RiiF.jpeg
paper_authors.fullname: Haonan Duan
paper_authors.type: user
paper_authors.user: robot-haonan
paper_authors._id: 67e3e1e20706b07bfb2713da
paper_authors.name: Hengjun Pu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:45:01.725Z
paper_authors._id: 648a1e44fe11ebd7489c289c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/WwMcD9PK0gxIu2I0n0QyD.jpeg
paper_authors.fullname: Hengjun Pu
paper_authors.type: user
paper_authors.user: MIASANMIA
paper_authors._id: 67e3e1e20706b07bfb2713db
paper_authors.name: Ronglei Tong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:45:07.673Z
paper_authors._id: 66b9a5bb32be421cd8538cd6
paper_authors.avatarUrl: /avatars/a1f7c0fe3ed4741017db713b4e6d47c8.svg
paper_authors.fullname: Ronglei Tong
paper_authors.type: user
paper_authors.user: TTTTTony
paper_authors._id: 67e3e1e20706b07bfb2713dc
paper_authors.name: Chengyang Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:45:13.906Z
paper_authors._id: 679165b9c7f527ef3619504e
paper_authors.avatarUrl: /avatars/f3e6ce5fc3d05c8632d8b208f55c2987.svg
paper_authors.fullname: Chengyang Zhao
paper_authors.type: user
paper_authors.user: chengyzhao
paper_authors._id: 67e3e1e20706b07bfb2713dd
paper_authors.name: Xizhou Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:45:20.614Z
paper_authors._id: 64ae2359179421d320b1694b
paper_authors.avatarUrl: /avatars/c387a75191005bcaa473091de5383a10.svg
paper_authors.fullname: Xizhou Zhu
paper_authors.type: user
paper_authors.user: Einsiedler
paper_authors._id: 67e3e1e20706b07bfb2713de
paper_authors.name: Yu Qiao
paper_authors._id: 67e3e1e20706b07bfb2713df
paper_authors.name: Jifeng Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:45:27.039Z
paper_authors._id: 64686f7172d9180d4ac8b4e4
paper_authors.avatarUrl: /avatars/db67dd6c4b2b41054ddcce5a18ade6f8.svg
paper_authors.fullname: Jifeng Dai
paper_authors.type: user
paper_authors.user: daijifeng
paper_authors._id: 67e3e1e20706b07bfb2713e0
paper_authors.name: Yuntao Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-27T09:45:40.200Z
paper_authors._id: 632dab84fdb35759ea6646a0
paper_authors.avatarUrl: /avatars/857b0b4d115aa5ab2f143e60b0e4edc6.svg
paper_authors.fullname: Yuntao Chen
paper_authors.type: user
paper_authors.user: YuntaoChen
paper_title: Dita: Scaling Diffusion Transformer for Generalist
    Vision-Language-Action Policy
paper_summary: While recent vision-language-action models trained on diverse robot datasets
  exhibit promising generalization capabilities with limited in-domain data,
  their reliance on compact action heads to predict discretized or continuous
  actions constrains adaptability to heterogeneous action spaces. We present
  Dita, a scalable framework that leverages Transformer architectures to directly
  denoise continuous action sequences through a unified multimodal diffusion
  process. Departing from prior methods that condition denoising on fused
  embeddings via shallow networks, Dita employs in-context conditioning --
  enabling fine-grained alignment between denoised actions and raw visual tokens
  from historical observations. This design explicitly models action deltas and
  environmental nuances. By scaling the diffusion action denoiser alongside the
  Transformer's scalability, Dita effectively integrates cross-embodiment
  datasets across diverse camera perspectives, observation scenes, tasks, and
  action spaces. Such synergy enhances robustness against various variances and
  facilitates the successful execution of long-horizon tasks. Evaluations across
  extensive benchmarks demonstrate state-of-the-art or comparative performance in
  simulation. Notably, Dita achieves robust real-world adaptation to
  environmental variances and complex long-horizon tasks through 10-shot
  finetuning, using only third-person camera inputs. The architecture establishes
  a versatile, lightweight and open-source baseline for generalist robot policy
  learning. Project Page: https://robodita.github.io.
paper_ai_summary: Dita, a scalable framework using Transformer architectures and multimodal diffusion, achieves state-of-the-art performance with fine-grained action denoising and real-world adaptation.
paper_ai_keywords: Transformer architectures multimodal diffusion denoising in-context conditioning action deltas cross-embodiment datasets long-horizon tasks 10-shot finetuning