Think or Not? Selective Reasoning via Reinforcement Learning for
  Vision-Language Models
Reinforcement Learning (RL) has proven to be an effective post-training
strategy for enhancing reasoning in vision-language models (VLMs). Group
Relative Policy Optimization (GRPO) is a recent prominent method that
encourages models to generate complete reasoning traces before answering,
leading to increased token usage and computational cost. Inspired by the
human-like thinking process-where people skip reasoning for easy questions but
think carefully when needed-we explore how to enable VLMs to first decide when
reasoning is necessary. To realize this, we propose TON, a two-stage training
strategy: (i) a supervised fine-tuning (SFT) stage with a simple yet effective
'thought dropout' operation, where reasoning traces are randomly replaced with
empty thoughts. This introduces a think-or-not format that serves as a cold
start for selective reasoning; (ii) a GRPO stage that enables the model to
freely explore when to think or not, while maximizing task-aware outcome
rewards. Experimental results show that TON can reduce the completion length by
up to 90% compared to vanilla GRPO, without sacrificing performance or even
improving it. Further evaluations across diverse vision-language tasks-covering
a range of reasoning difficulties under both 3B and 7B models-consistently
reveal that the model progressively learns to bypass unnecessary reasoning
steps as training advances. These findings shed light on the path toward
human-like reasoning patterns in reinforcement learning approaches. Our code is
available at https://github.com/kokolerk/TON.
682fdd8691757629e1d58e16
Jiaqi Wang
claimed_verified
2025-07-03T16:19:20.176Z
65b862af23d948d884ae9169
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/aAAokhlISNMiNuO_0LzrM.png
jiaqi wang
user
kolerk
682fdd8691757629e1d58e17
Kevin Qinghong Lin
claimed_verified
2025-05-26T08:16:35.662Z
64440be5af034cdfd69ca3a7
https://cdn-avatars.huggingface.co/v1/production/uploads/64440be5af034cdfd69ca3a7/qmx24QiDFT29vleCxL9TX.jpeg
Qinghong (Kevin) Lin
user
KevinQHLin
682fdd8691757629e1d58e18
James Cheng
682fdd8691757629e1d58e19
Mike Zheng Shou
Think or Not? Selective Reasoning via Reinforcement Learning for
  Vision-Language Models
Reinforcement Learning (RL) has proven to be an effective post-training
strategy for enhancing reasoning in vision-language models (VLMs). Group
Relative Policy Optimization (GRPO) is a recent prominent method that
encourages models to generate complete reasoning traces before answering,
leading to increased token usage and computational cost. Inspired by the
human-like thinking process-where people skip reasoning for easy questions but
think carefully when needed-we explore how to enable VLMs to first decide when
reasoning is necessary. To realize this, we propose TON, a two-stage training
strategy: (i) a supervised fine-tuning (SFT) stage with a simple yet effective
'thought dropout' operation, where reasoning traces are randomly replaced with
empty thoughts. This introduces a think-or-not format that serves as a cold
start for selective reasoning; (ii) a GRPO stage that enables the model to
freely explore when to think or not, while maximizing task-aware outcome
rewards. Experimental results show that TON can reduce the completion length by
up to 90% compared to vanilla GRPO, without sacrificing performance or even
improving it. Further evaluations across diverse vision-language tasks-covering
a range of reasoning difficulties under both 3B and 7B models-consistently
reveal that the model progressively learns to bypass unnecessary reasoning
steps as training advances. These findings shed light on the path toward
human-like reasoning patterns in reinforcement learning approaches. Our code is
available at https://github.com/kokolerk/TON.
TON, a two-stage training strategy combining supervised fine-tuning with thought dropout and Group Relative Policy Optimization, reduces unnecessary reasoning steps in vision-language models without sacrificing performance.
Reinforcement Learning (RL) vision-language models (VLMs) Group Relative Policy Optimization (GRPO) thought dropout selective reasoning