LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon
  Embodied Tasks
Real-world embodied agents face long-horizon tasks, characterized by
high-level goals demanding multi-step solutions beyond single actions.
Successfully navigating these requires both high-level task planning (i.e.,
decomposing goals into sub-tasks) and low-level motion control (i.e.,
generating precise robot actions). While existing vision language action (VLA)
models and hierarchical architectures offer potential in embodied tasks, the
former often falter in planning, and the latter can suffer from coordination
issues, both hampering performance. We introduce a new unified VLA framework
for long-horizon tasks, dubbed LoHoVLA, to overcome these limitations. LoHoVLA
leverages a large pretrained vision language model (VLM) as the backbone to
jointly generate language and action tokens for sub-task generation and robot
action prediction, respectively. This shared representation promotes better
generalization across tasks. Additionally, LoHoVLA embraces a hierarchical
closed-loop control mechanism to mitigate errors originating from both
high-level planning and low-level control. To train LoHoVLA, we introduce
LoHoSet, a dataset built on the Ravens simulator, containing 20 long-horizon
tasks, each with 1,000 expert demonstrations composed of visual observations,
linguistic goals, sub-tasks, and robot actions. Experimental results show that
LoHoVLA significantly surpasses both hierarchical and standard VLA approaches
on long-horizon embodied tasks in the Ravens simulator. These findings
underscore the promise of unified architectures for advancing generalizable
embodied intelligence.
683e86e31bca54bb6d169fc4
Yi Yang
claimed_verified
2025-06-03T08:40:09.713Z
6707eaaf5f50b17754ff9cbc
/avatars/f08ca6228b124a8955787d0662a52bbd.svg
Yang
user
Yysrc
683e86e31bca54bb6d169fc5
Jiaxuan Sun
683e86e31bca54bb6d169fc6
Siqi Kou
admin_assigned
2025-06-03T12:35:37.198Z
654e330f350abceb30a1390b
/avatars/e54a8be788fa1bdc7acefecc208215bb.svg
KouSiqi
user
karrykkk
683e86e31bca54bb6d169fc7
Yihan Wang
683e86e31bca54bb6d169fc8
Zhijie Deng
admin_assigned
2025-06-03T12:35:13.092Z
673d5f411b0fe168ad4896b2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/fYgQx6XNi_P5GxlUKbH5G.png
Zhijie Deng
user
thudzj
LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon
  Embodied Tasks
Real-world embodied agents face long-horizon tasks, characterized by
high-level goals demanding multi-step solutions beyond single actions.
Successfully navigating these requires both high-level task planning (i.e.,
decomposing goals into sub-tasks) and low-level motion control (i.e.,
generating precise robot actions). While existing vision language action (VLA)
models and hierarchical architectures offer potential in embodied tasks, the
former often falter in planning, and the latter can suffer from coordination
issues, both hampering performance. We introduce a new unified VLA framework
for long-horizon tasks, dubbed LoHoVLA, to overcome these limitations. LoHoVLA
leverages a large pretrained vision language model (VLM) as the backbone to
jointly generate language and action tokens for sub-task generation and robot
action prediction, respectively. This shared representation promotes better
generalization across tasks. Additionally, LoHoVLA embraces a hierarchical
closed-loop control mechanism to mitigate errors originating from both
high-level planning and low-level control. To train LoHoVLA, we introduce
LoHoSet, a dataset built on the Ravens simulator, containing 20 long-horizon
tasks, each with 1,000 expert demonstrations composed of visual observations,
linguistic goals, sub-tasks, and robot actions. Experimental results show that
LoHoVLA significantly surpasses both hierarchical and standard VLA approaches
on long-horizon embodied tasks in the Ravens simulator. These findings
underscore the promise of unified architectures for advancing generalizable
embodied intelligence.
A unified vision language action framework, LoHoVLA, combines a large pretrained vision language model with hierarchical closed-loop control to improve performance on long-horizon embodied tasks.
vision language action models hierarchical architectures high-level task planning low-level motion control LoHoVLA large pretrained vision language model shared representation hierarchical closed-loop control LoHoSet Ravens simulator long-horizon tasks sub-task generation robot action prediction embodied intelligence