Improved Visual-Spatial Reasoning via R1-Zero-Like Training
Increasing attention has been placed on improving the reasoning capacities of
multi-modal large language models (MLLMs). As the cornerstone for AI agents
that function in the physical realm, video-based visual-spatial intelligence
(VSI) emerges as one of the most pivotal reasoning capabilities of MLLMs. This
work conducts a first, in-depth study on improving the visual-spatial reasoning
of MLLMs via R1-Zero-like training. Technically, we first identify that the
visual-spatial reasoning capacities of small- to medium-sized Qwen2-VL models
cannot be activated via Chain of Thought (CoT) prompts. We then incorporate
GRPO training for improved visual-spatial reasoning, using the carefully
curated VSI-100k dataset, following DeepSeek-R1-Zero. During the investigation,
we identify the necessity to keep the KL penalty (even with a small value) in
GRPO. With just 120 GPU hours, our vsGRPO-2B model, fine-tuned from
Qwen2-VL-2B, can outperform the base model by 12.1% and surpass GPT-4o.
Moreover, our vsGRPO-7B model, fine-tuned from Qwen2-VL-7B, achieves
performance comparable to that of the best open-source model
LLaVA-NeXT-Video-72B. Additionally, we compare vsGRPO to supervised fine-tuning
and direct preference optimization baselines and observe strong performance
superiority. The code and dataset will be available soon.
67edf28e042e8ba3e95d1960
Zhenyi Liao
admin_assigned
2025-04-03T13:44:50.157Z
67ed94169723a841a7128c6f
/avatars/950397084cc48c0885cf85bd3e5260b1.svg
ZhenYi Liao
user
ZhenYiao
67edf28e042e8ba3e95d1961
Qingsong Xie
admin_assigned
2025-04-03T13:45:08.446Z
64184911db24526c7c9ca035
/avatars/2be15485bdd9e00857d4bff107d1577e.svg
qingsong xie
user
kongming11
67edf28e042e8ba3e95d1962
Yanhao Zhang
admin_assigned
2025-04-03T13:45:23.767Z
6527849a5fc08c60dc6a3120
/avatars/54651dc48acd119c012520a616285e7a.svg
Yanhao Zhang
user
YanhaoZhang
67edf28e042e8ba3e95d1963
Zijian Kong
67edf28e042e8ba3e95d1964
Haonan Lu
admin_assigned
2025-04-03T13:45:39.183Z
603864181b642aef9350af81
/avatars/27ec0b885cbedef007876c8c98c69d15.svg
Haonan Lu
user
luhaonan20
67edf28e042e8ba3e95d1965
Zhenyu Yang
67edf28e042e8ba3e95d1966
Zhijie Deng
claimed_verified
2025-04-03T07:52:14.204Z
64bba541da140e461924dfed
/avatars/367993765b0ca3734b2b100db33ed787.svg
zhijie deng
user
zhijie3
Improved Visual-Spatial Reasoning via R1-Zero-Like Training
Increasing attention has been placed on improving the reasoning capacities of
multi-modal large language models (MLLMs). As the cornerstone for AI agents
that function in the physical realm, video-based visual-spatial intelligence
(VSI) emerges as one of the most pivotal reasoning capabilities of MLLMs. This
work conducts a first, in-depth study on improving the visual-spatial reasoning
of MLLMs via R1-Zero-like training. Technically, we first identify that the
visual-spatial reasoning capacities of small- to medium-sized Qwen2-VL models
cannot be activated via Chain of Thought (CoT) prompts. We then incorporate
GRPO training for improved visual-spatial reasoning, using the carefully
curated VSI-100k dataset, following DeepSeek-R1-Zero. During the investigation,
we identify the necessity to keep the KL penalty (even with a small value) in
GRPO. With just 120 GPU hours, our vsGRPO-2B model, fine-tuned from
Qwen2-VL-2B, can outperform the base model by 12.1% and surpass GPT-4o.
Moreover, our vsGRPO-7B model, fine-tuned from Qwen2-VL-7B, achieves
performance comparable to that of the best open-source model
LLaVA-NeXT-Video-72B. Additionally, we compare vsGRPO to supervised fine-tuning
and direct preference optimization baselines and observe strong performance
superiority. The code and dataset will be available soon.
The study enhances visual-spatial reasoning in multi-modal large language models through GRPO training using the VSI-100k dataset, demonstrating significant performance improvements over base models.
multi-modal large language models visual-spatial intelligence R1-Zero-like training Chain of Thought GRPO KL penalty VSI-100k vsGRPO Qwen2-VL GPT-4 LLaVA-NeXT-Video supervised fine-tuning direct preference optimization