title: Rethinking RL Scaling for Vision Language Models: A Transparent,
    From-Scratch Framework and Comprehensive Evaluation Scheme
summary: Reinforcement learning (RL) has recently shown strong potential in improving
  the reasoning capabilities of large language models and is now being actively
  extended to vision-language models (VLMs). However, existing RL applications in
  VLMs often rely on heavily engineered frameworks that hinder reproducibility
  and accessibility, while lacking standardized evaluation protocols, making it
  difficult to compare results or interpret training dynamics. This work
  introduces a transparent, from-scratch framework for RL in VLMs, offering a
  minimal yet functional four-step pipeline validated across multiple models and
  datasets. In addition, a standardized evaluation scheme is proposed to assess
  training dynamics and reflective behaviors. Extensive experiments on visual
  reasoning tasks uncover key empirical findings: response length is sensitive to
  random seeds, reflection correlates with output length, and RL consistently
  outperforms supervised fine-tuning (SFT) in generalization, even with
  high-quality data. These findings, together with the proposed framework, aim to
  establish a reproducible baseline and support broader engagement in RL-based
  VLM research.
paper_authors._id: 67ef3f9804be7fba0c882738
paper_authors.name: Yan Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-04T07:17:53.820Z
paper_authors._id: 633fc70529b5a95f6e15a6b7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg
paper_authors.fullname: Yan Ma
paper_authors.type: user
paper_authors.user: ManTle
paper_authors._id: 67ef3f9804be7fba0c882739
paper_authors.name: Steffi Chern
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-04T07:09:14.660Z
paper_authors._id: 64b370fe6d953e7c75ede314
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b370fe6d953e7c75ede314/RdP2q3hGXWE4E2zfSv0KU.png
paper_authors.fullname: Steffi Chern
paper_authors.type: user
paper_authors.user: steffichern
paper_authors._id: 67ef3f9804be7fba0c88273a
paper_authors.name: Xuyang Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-04T07:17:40.397Z
paper_authors._id: 642e4d4d6748dd4f8eeb7732
paper_authors.avatarUrl: /avatars/fd911e9143d1a7aedd21a7d611543fcc.svg
paper_authors.fullname: Xuyang Shen
paper_authors.type: user
paper_authors.user: Ryan1122
paper_authors._id: 67ef3f9804be7fba0c88273b
paper_authors.name: Yiran Zhong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-04T07:09:16.707Z
paper_authors._id: 64c525e4d68946edad6c7067
paper_authors.avatarUrl: /avatars/1b108661634af602717a4ab4b66a151f.svg
paper_authors.fullname: Yiran Zhong
paper_authors.type: user
paper_authors.user: IanZhong
paper_authors._id: 67ef3f9804be7fba0c88273c
paper_authors.name: Pengfei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-04T07:17:34.472Z
paper_authors._id: 6144a0c4ff1146bbd84d9865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
paper_authors.fullname: Pengfei Liu
paper_authors.type: user
paper_authors.user: Pengfei
paper_title: Rethinking RL Scaling for Vision Language Models: A Transparent,
    From-Scratch Framework and Comprehensive Evaluation Scheme
paper_summary: Reinforcement learning (RL) has recently shown strong potential in improving
  the reasoning capabilities of large language models and is now being actively
  extended to vision-language models (VLMs). However, existing RL applications in
  VLMs often rely on heavily engineered frameworks that hinder reproducibility
  and accessibility, while lacking standardized evaluation protocols, making it
  difficult to compare results or interpret training dynamics. This work
  introduces a transparent, from-scratch framework for RL in VLMs, offering a
  minimal yet functional four-step pipeline validated across multiple models and
  datasets. In addition, a standardized evaluation scheme is proposed to assess
  training dynamics and reflective behaviors. Extensive experiments on visual
  reasoning tasks uncover key empirical findings: response length is sensitive to
  random seeds, reflection correlates with output length, and RL consistently
  outperforms supervised fine-tuning (SFT) in generalization, even with
  high-quality data. These findings, together with the proposed framework, aim to
  establish a reproducible baseline and support broader engagement in RL-based
  VLM research.
paper_ai_summary: A transparent reinforcement learning framework and standardized evaluation are introduced for vision-language models, showing RL's superiority in generalization over supervised fine-tuning.
paper_ai_keywords: reinforcement learning vision-language models reproducibility evaluation protocols training dynamics reflection output length supervised fine-tuning