UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
A generalist robot should perform effectively across various environments.
However, most existing approaches heavily rely on scaling action-annotated data
to enhance their capabilities. Consequently, they are often limited to single
physical specification and struggle to learn transferable knowledge across
different embodiments and environments. To confront these limitations, we
propose UniVLA, a new framework for learning cross-embodiment
vision-language-action (VLA) policies. Our key innovation is to derive
task-centric action representations from videos with a latent action model.
This enables us to exploit extensive data across a wide spectrum of embodiments
and perspectives. To mitigate the effect of task-irrelevant dynamics, we
incorporate language instructions and establish a latent action model within
the DINO feature space. Learned from internet-scale videos, the generalist
policy can be deployed to various robots through efficient latent action
decoding. We obtain state-of-the-art results across multiple manipulation and
navigation benchmarks, as well as real-robot deployments. UniVLA achieves
superior performance over OpenVLA with less than 1/20 of pretraining compute
and 1/10 of downstream data. Continuous performance improvements are observed
as heterogeneous data, even including human videos, are incorporated into the
training pipeline. The results underscore UniVLA's potential to facilitate
scalable and efficient robot policy learning.
68218b847202d193249511b6
Qingwen Bu
admin_assigned
2025-05-12T13:14:55.391Z
64ac1f169dcc5787461468a4
/avatars/c031a75989147009b7850df4eddfcb27.svg
Qingwen Bu
user
qwbu
68218b847202d193249511b7
Yanting Yang
68218b847202d193249511b8
Jisong Cai
admin_assigned
2025-05-12T13:15:26.413Z
66a3402e4c2093e582bdf511
/avatars/6f2e1f37b6a6cf9dc6df228482c0777a.svg
Jisong Cai
user
SereneC
68218b847202d193249511b9
Shenyuan Gao
admin_assigned
2025-05-12T13:15:33.171Z
654a31c073416a223f3b5fca
/avatars/bab382c46787eaf7889ed241e12775ee.svg
Shenyuan Gao
user
Little-Podi
68218b847202d193249511ba
Guanghui Ren
claimed_verified
2025-05-12T06:50:15.305Z
646ec9b135f55eb49e405faa
/avatars/a17194be585d20e2a021e77a5a20e213.svg
Guanghui Ren
user
sundrops
68218b847202d193249511bb
Maoqing Yao
admin_assigned
2025-05-12T13:15:39.850Z
67739bfa64e8b7438ae68eb4
/avatars/15193bfbce487b2de4ce8c86bd18885a.svg
Maoqing Yao
user
AutobotZero
68218b847202d193249511bc
Ping Luo
admin_assigned
2025-05-12T13:15:47.622Z
67cb7d55560c3dcbb1adeaa3
/avatars/0b616d3655b0b54a621c2608b2f14379.svg
Ping Luo
user
appleluo
68218b847202d193249511bd
Hongyang Li
admin_assigned
2025-05-12T13:16:21.369Z
6499b0184936457997180c90
/avatars/b8be7bfabf746639e30330f5f623f560.svg
Hongyang Li
user
compileme
UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
A generalist robot should perform effectively across various environments.
However, most existing approaches heavily rely on scaling action-annotated data
to enhance their capabilities. Consequently, they are often limited to single
physical specification and struggle to learn transferable knowledge across
different embodiments and environments. To confront these limitations, we
propose UniVLA, a new framework for learning cross-embodiment
vision-language-action (VLA) policies. Our key innovation is to derive
task-centric action representations from videos with a latent action model.
This enables us to exploit extensive data across a wide spectrum of embodiments
and perspectives. To mitigate the effect of task-irrelevant dynamics, we
incorporate language instructions and establish a latent action model within
the DINO feature space. Learned from internet-scale videos, the generalist
policy can be deployed to various robots through efficient latent action
decoding. We obtain state-of-the-art results across multiple manipulation and
navigation benchmarks, as well as real-robot deployments. UniVLA achieves
superior performance over OpenVLA with less than 1/20 of pretraining compute
and 1/10 of downstream data. Continuous performance improvements are observed
as heterogeneous data, even including human videos, are incorporated into the
training pipeline. The results underscore UniVLA's potential to facilitate
scalable and efficient robot policy learning.
UniVLA is a framework for learning cross-embodiment VLA policies using latent action models derived from internet-scale videos, achieving superior performance with reduced pretraining compute and downstream data.
latent action model VLA policies DINO feature space