G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language
  Model via Reinforcement Learning
Vision-Language Models (VLMs) excel in many direct multimodal tasks but
struggle to translate this prowess into effective decision-making within
interactive, visually rich environments like games. This ``knowing-doing'' gap
significantly limits their potential as autonomous agents, as leading VLMs
often performing badly in simple games. To address this, we introduce VLM-Gym,
a curated reinforcement learning (RL) environment featuring diverse visual
games with unified interfaces and adjustable, compositional difficulty,
specifically designed for scalable multi-game parallel training. Leveraging
VLM-Gym, we train G0 models using pure RL-driven self-evolution, which
demonstrate emergent perception and reasoning patterns. To further mitigate
challenges arising from game diversity, we develop G1 models. G1 incorporates a
perception-enhanced cold start prior to RL fine-tuning. Our resulting G1 models
consistently surpass their teacher across all games and outperform leading
proprietary models like Claude-3.7-Sonnet-Thinking. Systematic analysis reveals
an intriguing finding: perception and reasoning abilities mutually bootstrap
each other throughout the RL training process. Source code including VLM-Gym
and RL training are released at https://github.com/chenllliang/G1 to foster
future research in advancing VLMs as capable interactive agents.
682c641925f124206513d14d
Liang Chen
admin_assigned
2025-05-27T13:03:52.706Z
61b0a4ce1b3d95b3d1ed9251
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Wwjr26vdudX5KYVTb8Q0a.png
Liang Chen
user
leonardPKU
682c641925f124206513d14e
Hongcheng Gao
admin_assigned
2025-05-27T13:04:07.611Z
62728f4f6253fe2068da1021
https://cdn-avatars.huggingface.co/v1/production/uploads/62728f4f6253fe2068da1021/KZ65X0EH98AF3zXemPiap.jpeg
Hongcheng Gao
user
HongchengGao
682c641925f124206513d14f
Tianyu Liu
682c641925f124206513d150
Zhiqi Huang
admin_assigned
2025-05-27T13:04:23.439Z
66221f1a90f3fd333c4ec52e
/avatars/a3173d9603a69020ec24170831c97c2f.svg
Zhiqi Huang
user
Angelalilyer
682c641925f124206513d151
Flood Sung
admin_assigned
2025-05-27T13:04:29.419Z
6343d01a08c017b2c042305d
https://cdn-avatars.huggingface.co/v1/production/uploads/6343d01a08c017b2c042305d/cmJrYkGs9RjDAKfMYCqdW.jpeg
Flood Sung
user
floodsung
682c641925f124206513d152
Xinyu Zhou
admin_assigned
2025-05-27T13:04:35.487Z
6465adc302a171e703b3f093
/avatars/639bdb4bc382633e40ec7ec78e6a2509.svg
Xinyu Zhou
user
xinyuzhou
682c641925f124206513d153
Yuxin Wu
682c641925f124206513d154
Baobao Chang
G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language
  Model via Reinforcement Learning
Vision-Language Models (VLMs) excel in many direct multimodal tasks but
struggle to translate this prowess into effective decision-making within
interactive, visually rich environments like games. This ``knowing-doing'' gap
significantly limits their potential as autonomous agents, as leading VLMs
often performing badly in simple games. To address this, we introduce VLM-Gym,
a curated reinforcement learning (RL) environment featuring diverse visual
games with unified interfaces and adjustable, compositional difficulty,
specifically designed for scalable multi-game parallel training. Leveraging
VLM-Gym, we train G0 models using pure RL-driven self-evolution, which
demonstrate emergent perception and reasoning patterns. To further mitigate
challenges arising from game diversity, we develop G1 models. G1 incorporates a
perception-enhanced cold start prior to RL fine-tuning. Our resulting G1 models
consistently surpass their teacher across all games and outperform leading
proprietary models like Claude-3.7-Sonnet-Thinking. Systematic analysis reveals
an intriguing finding: perception and reasoning abilities mutually bootstrap
each other throughout the RL training process. Source code including VLM-Gym
and RL training are released at https://github.com/chenllliang/G1 to foster
future research in advancing VLMs as capable interactive agents.
VLM-Gym addresses the "knowing-doing" gap in Vision-Language Models by training them in a diverse RL environment, leading to enhanced perception and reasoning abilities that surpass existing models in interactive games.
Vision-Language Models VLM-Gym reinforcement learning RL G0 models self-evolution G1 models perception-enhanced cold start RL fine-tuning