BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation
Vision-Language-Action (VLA) models have shown impressive capabilities across
a wide range of robotics manipulation tasks. However, their growing model size
poses significant challenges for deployment on resource-constrained robotic
systems. While 1-bit pretraining has proven effective for enhancing the
inference efficiency of large language models with minimal performance loss,
its application to VLA models remains underexplored. In this work, we present
BitVLA, the first 1-bit VLA model for robotics manipulation, in which every
parameter is ternary, i.e., {-1, 0, 1}. To further reduce the memory footprint
of the vision encoder, we propose the distillation-aware training strategy that
compresses the full-precision encoder to 1.58-bit weights. During this process,
a full-precision encoder serves as a teacher model to better align latent
representations. Despite the lack of large-scale robotics pretraining, BitVLA
achieves performance comparable to the state-of-the-art model OpenVLA-OFT with
4-bit post-training quantization on the LIBERO benchmark, while consuming only
29.8% of the memory. These results highlight BitVLA's promise for deployment on
memory-constrained edge devices. We release the code and model weights in
https://github.com/ustcwhy/BitVLA.
68478dae3ec10bdd8ab4dd9b
Hongyu Wang
claimed_verified
2025-07-03T07:42:02.121Z
63f71771d36951307fcb4dcd
https://cdn-avatars.huggingface.co/v1/production/uploads/63f71771d36951307fcb4dcd/1c-GSQmSSuDXyVWjxZFy_.jpeg
Hongyu Wang
user
hongyuw
68478dae3ec10bdd8ab4dd9c
Chuyan Xiong
68478dae3ec10bdd8ab4dd9d
Ruiping Wang
68478dae3ec10bdd8ab4dd9e
Xilin Chen
BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation
Vision-Language-Action (VLA) models have shown impressive capabilities across
a wide range of robotics manipulation tasks. However, their growing model size
poses significant challenges for deployment on resource-constrained robotic
systems. While 1-bit pretraining has proven effective for enhancing the
inference efficiency of large language models with minimal performance loss,
its application to VLA models remains underexplored. In this work, we present
BitVLA, the first 1-bit VLA model for robotics manipulation, in which every
parameter is ternary, i.e., {-1, 0, 1}. To further reduce the memory footprint
of the vision encoder, we propose the distillation-aware training strategy that
compresses the full-precision encoder to 1.58-bit weights. During this process,
a full-precision encoder serves as a teacher model to better align latent
representations. Despite the lack of large-scale robotics pretraining, BitVLA
achieves performance comparable to the state-of-the-art model OpenVLA-OFT with
4-bit post-training quantization on the LIBERO benchmark, while consuming only
29.8% of the memory. These results highlight BitVLA's promise for deployment on
memory-constrained edge devices. We release the code and model weights in
https://github.com/ustcwhy/BitVLA.
BitVLA, a 1-bit VLA model with ternary parameters, achieves comparable performance to OpenVLA-OFT on LIBERO while using 29.8% less memory through distillation-aware training.
VLA models 1-bit pretraining ternary parameters distillation-aware training vision encoder full-precision encoder latent representations memory footprint robotics manipulation OpenVLA-OFT LIBERO benchmark memory-constrained edge devices