title: Think or Not? Selective Reasoning via Reinforcement Learning for
    Vision-Language Models
summary: Reinforcement Learning (RL) has proven to be an effective post-training
  strategy for enhancing reasoning in vision-language models (VLMs). Group
  Relative Policy Optimization (GRPO) is a recent prominent method that
  encourages models to generate complete reasoning traces before answering,
  leading to increased token usage and computational cost. Inspired by the
  human-like thinking process-where people skip reasoning for easy questions but
  think carefully when needed-we explore how to enable VLMs to first decide when
  reasoning is necessary. To realize this, we propose TON, a two-stage training
  strategy: (i) a supervised fine-tuning (SFT) stage with a simple yet effective
  'thought dropout' operation, where reasoning traces are randomly replaced with
  empty thoughts. This introduces a think-or-not format that serves as a cold
  start for selective reasoning; (ii) a GRPO stage that enables the model to
  freely explore when to think or not, while maximizing task-aware outcome
  rewards. Experimental results show that TON can reduce the completion length by
  up to 90% compared to vanilla GRPO, without sacrificing performance or even
  improving it. Further evaluations across diverse vision-language tasks-covering
  a range of reasoning difficulties under both 3B and 7B models-consistently
  reveal that the model progressively learns to bypass unnecessary reasoning
  steps as training advances. These findings shed light on the path toward
  human-like reasoning patterns in reinforcement learning approaches. Our code is
  available at https://github.com/kokolerk/TON.
paper_authors._id: 682fdd8691757629e1d58e16
paper_authors.name: Jiaqi Wang
paper_authors._id: 682fdd8691757629e1d58e17
paper_authors.name: Kevin Qinghong Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:16:35.662Z
paper_authors._id: 64440be5af034cdfd69ca3a7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64440be5af034cdfd69ca3a7/qmx24QiDFT29vleCxL9TX.jpeg
paper_authors.fullname: Qinghong (Kevin) Lin
paper_authors.type: user
paper_authors.user: KevinQHLin
paper_authors._id: 682fdd8691757629e1d58e18
paper_authors.name: James Cheng
paper_authors._id: 682fdd8691757629e1d58e19
paper_authors.name: Mike Zheng Shou
paper_title: Think or Not? Selective Reasoning via Reinforcement Learning for
    Vision-Language Models
paper_summary: Reinforcement Learning (RL) has proven to be an effective post-training
  strategy for enhancing reasoning in vision-language models (VLMs). Group
  Relative Policy Optimization (GRPO) is a recent prominent method that
  encourages models to generate complete reasoning traces before answering,
  leading to increased token usage and computational cost. Inspired by the
  human-like thinking process-where people skip reasoning for easy questions but
  think carefully when needed-we explore how to enable VLMs to first decide when
  reasoning is necessary. To realize this, we propose TON, a two-stage training
  strategy: (i) a supervised fine-tuning (SFT) stage with a simple yet effective
  'thought dropout' operation, where reasoning traces are randomly replaced with
  empty thoughts. This introduces a think-or-not format that serves as a cold
  start for selective reasoning; (ii) a GRPO stage that enables the model to
  freely explore when to think or not, while maximizing task-aware outcome
  rewards. Experimental results show that TON can reduce the completion length by
  up to 90% compared to vanilla GRPO, without sacrificing performance or even
  improving it. Further evaluations across diverse vision-language tasks-covering
  a range of reasoning difficulties under both 3B and 7B models-consistently
  reveal that the model progressively learns to bypass unnecessary reasoning
  steps as training advances. These findings shed light on the path toward
  human-like reasoning patterns in reinforcement learning approaches. Our code is
  available at https://github.com/kokolerk/TON.
paper_ai_summary: TON, a two-stage training strategy combining supervised fine-tuning with thought dropout and Group Relative Policy Optimization, reduces unnecessary reasoning steps in vision-language models without sacrificing performance.
paper_ai_keywords: Reinforcement Learning (RL) vision-language models (VLMs) Group Relative Policy Optimization (GRPO) thought dropout selective reasoning