title: OpenVLA: An Open-Source Vision-Language-Action Model
summary: Large policies pretrained on a combination of Internet-scale vision-language
  data and diverse robot demonstrations have the potential to change how we teach
  robots new skills: rather than training new behaviors from scratch, we can
  fine-tune such vision-language-action (VLA) models to obtain robust,
  generalizable policies for visuomotor control. Yet, widespread adoption of VLAs
  for robotics has been challenging as 1) existing VLAs are largely closed and
  inaccessible to the public, and 2) prior work fails to explore methods for
  efficiently fine-tuning VLAs for new tasks, a key component for adoption.
  Addressing these challenges, we introduce OpenVLA, a 7B-parameter open-source
  VLA trained on a diverse collection of 970k real-world robot demonstrations.
  OpenVLA builds on a Llama 2 language model combined with a visual encoder that
  fuses pretrained features from DINOv2 and SigLIP. As a product of the added
  data diversity and new model components, OpenVLA demonstrates strong results
  for generalist manipulation, outperforming closed models such as RT-2-X (55B)
  by 16.5% in absolute task success rate across 29 tasks and multiple robot
  embodiments, with 7x fewer parameters. We further show that we can effectively
  fine-tune OpenVLA for new settings, with especially strong generalization
  results in multi-task environments involving multiple objects and strong
  language grounding abilities, and outperform expressive from-scratch imitation
  learning methods such as Diffusion Policy by 20.4%. We also explore compute
  efficiency; as a separate contribution, we show that OpenVLA can be fine-tuned
  on consumer GPUs via modern low-rank adaptation methods and served efficiently
  via quantization without a hit to downstream success rate. Finally, we release
  model checkpoints, fine-tuning notebooks, and our PyTorch codebase with
  built-in support for training VLAs at scale on Open X-Embodiment datasets.
paper_authors._id: 666b9d34cf416fdc65f87643
paper_authors.name: Moo Jin Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T07:20:40.563Z
paper_authors._id: 64b017833ffa69983bb8ed37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b017833ffa69983bb8ed37/gdaFjlaJDDPtfH84b75RU.jpeg
paper_authors.fullname: Moo Jin Kim
paper_authors.type: user
paper_authors.user: moojink
paper_authors._id: 666b9d34cf416fdc65f87644
paper_authors.name: Karl Pertsch
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T07:20:48.705Z
paper_authors._id: 65d4c1ff29b4ac81c265e6e6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65d4c1ff29b4ac81c265e6e6/GXgs28okxGfpBdqhxov9-.png
paper_authors.fullname: Karl Pertsch
paper_authors.type: user
paper_authors.user: KarlP
paper_authors._id: 666b9d34cf416fdc65f87645
paper_authors.name: Siddharth Karamcheti
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T07:20:55.211Z
paper_authors._id: 6150b090d84cf0532aa1764b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634666324094-6150b090d84cf0532aa1764b.jpeg
paper_authors.fullname: Siddharth Karamcheti
paper_authors.type: user
paper_authors.user: skaramcheti
paper_authors._id: 666b9d34cf416fdc65f87646
paper_authors.name: Ted Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T07:21:02.137Z
paper_authors._id: 64246a35089d5fae56b4a584
paper_authors.avatarUrl: /avatars/bc2f1f1135dbc3460ae30b0e7a0cf39b.svg
paper_authors.fullname: xiao
paper_authors.type: user
paper_authors.user: tedxiao
paper_authors._id: 666b9d34cf416fdc65f87647
paper_authors.name: Ashwin Balakrishna
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:04:21.971Z
paper_authors._id: 653ad464170f0a2a9202dbe3
paper_authors.avatarUrl: /avatars/dd48cf05b946663dcb927545cd8edddc.svg
paper_authors.fullname: Ashwin Balakrishna
paper_authors.type: user
paper_authors.user: abalakrishnaTRI
paper_authors._id: 666b9d34cf416fdc65f87648
paper_authors.name: Suraj Nair
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:04:40.855Z
paper_authors._id: 624f0e38c8b211c193d325dd
paper_authors.avatarUrl: /avatars/370b7b21bd1bc284d3f48c0c40dfa026.svg
paper_authors.fullname: Suraj Nair
paper_authors.type: user
paper_authors.user: surajnair
paper_authors._id: 666b9d34cf416fdc65f87649
paper_authors.name: Rafael Rafailov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:04:47.337Z
paper_authors._id: 64701c98d742e9ef6522598f
paper_authors.avatarUrl: /avatars/df3a52bd4bc374d7cb4061f449f75c07.svg
paper_authors.fullname: Rafael Rafailov
paper_authors.type: user
paper_authors.user: rmrafailov
paper_authors._id: 666b9d34cf416fdc65f8764a
paper_authors.name: Ethan Foster
paper_authors._id: 666b9d34cf416fdc65f8764b
paper_authors.name: Grace Lam
paper_authors._id: 666b9d34cf416fdc65f8764c
paper_authors.name: Pannag Sanketi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-09T07:48:52.353Z
paper_authors._id: 66b56f73fc66a466e6104bfb
paper_authors.avatarUrl: /avatars/74855aaf580144fafead4b93d6a24a46.svg
paper_authors.fullname: Pannag Sanketi
paper_authors.type: user
paper_authors.user: prsanky
paper_authors._id: 666b9d34cf416fdc65f8764d
paper_authors.name: Quan Vuong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:05:19.625Z
paper_authors._id: 63e1d6687fbb6ae4d4f4d980
paper_authors.avatarUrl: /avatars/4959319e69d69e03a31ff4be326e602a.svg
paper_authors.fullname: Quan Vuong
paper_authors.type: user
paper_authors.user: kabut1211
paper_authors._id: 666b9d34cf416fdc65f8764e
paper_authors.name: Thomas Kollar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:05:34.367Z
paper_authors._id: 64358e662d0ed796668cb438
paper_authors.avatarUrl: /avatars/6f86547d8691e061e289dd906deedfdb.svg
paper_authors.fullname: Thomas Kollar
paper_authors.type: user
paper_authors.user: thomas-kollar
paper_authors._id: 666b9d34cf416fdc65f8764f
paper_authors.name: Benjamin Burchfiel
paper_authors._id: 666b9d34cf416fdc65f87650
paper_authors.name: Russ Tedrake
paper_authors._id: 666b9d34cf416fdc65f87651
paper_authors.name: Dorsa Sadigh
paper_authors._id: 666b9d34cf416fdc65f87652
paper_authors.name: Sergey Levine
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:05:50.110Z
paper_authors._id: 665ce54120a307a3754849dd
paper_authors.avatarUrl: /avatars/e698726e9be61dd50ce2efe372ed5dac.svg
paper_authors.fullname: Sergey Levine
paper_authors.type: user
paper_authors.user: svlevine
paper_authors._id: 666b9d34cf416fdc65f87653
paper_authors.name: Percy Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:05:55.774Z
paper_authors._id: 6409651b9e9f790c905b2335
paper_authors.avatarUrl: /avatars/1fb8c80b60f21f65a0a027319101f236.svg
paper_authors.fullname: Percy Liang
paper_authors.type: user
paper_authors.user: percyliang
paper_authors._id: 666b9d34cf416fdc65f87654
paper_authors.name: Chelsea Finn
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-14T10:06:01.500Z
paper_authors._id: 64ac22a9193f0a807deb673d
paper_authors.avatarUrl: /avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
paper_authors.fullname: Chelsea Finn
paper_authors.type: user
paper_authors.user: cbfinn
paper_title: OpenVLA: An Open-Source Vision-Language-Action Model
paper_summary: Large policies pretrained on a combination of Internet-scale vision-language
  data and diverse robot demonstrations have the potential to change how we teach
  robots new skills: rather than training new behaviors from scratch, we can
  fine-tune such vision-language-action (VLA) models to obtain robust,
  generalizable policies for visuomotor control. Yet, widespread adoption of VLAs
  for robotics has been challenging as 1) existing VLAs are largely closed and
  inaccessible to the public, and 2) prior work fails to explore methods for
  efficiently fine-tuning VLAs for new tasks, a key component for adoption.
  Addressing these challenges, we introduce OpenVLA, a 7B-parameter open-source
  VLA trained on a diverse collection of 970k real-world robot demonstrations.
  OpenVLA builds on a Llama 2 language model combined with a visual encoder that
  fuses pretrained features from DINOv2 and SigLIP. As a product of the added
  data diversity and new model components, OpenVLA demonstrates strong results
  for generalist manipulation, outperforming closed models such as RT-2-X (55B)
  by 16.5% in absolute task success rate across 29 tasks and multiple robot
  embodiments, with 7x fewer parameters. We further show that we can effectively
  fine-tune OpenVLA for new settings, with especially strong generalization
  results in multi-task environments involving multiple objects and strong
  language grounding abilities, and outperform expressive from-scratch imitation
  learning methods such as Diffusion Policy by 20.4%. We also explore compute
  efficiency; as a separate contribution, we show that OpenVLA can be fine-tuned
  on consumer GPUs via modern low-rank adaptation methods and served efficiently
  via quantization without a hit to downstream success rate. Finally, we release
  model checkpoints, fine-tuning notebooks, and our PyTorch codebase with
  built-in support for training VLAs at scale on Open X-Embodiment datasets.
paper_ai_keywords: Llama 2 DINOv2 SigLIP OpenVLA generalist manipulation RT-2-X multi-task environments language grounding Diffusion Policy low-rank adaptation quantization Open X-Embodiment datasets PyTorch