title: Missing Premise exacerbates Overthinking: Are Reasoning Models losing
    Critical Thinking Skill?
summary: We find that the response length of reasoning LLMs, whether trained by
  reinforcement learning or supervised learning, drastically increases for
  ill-posed questions with missing premises (MiP), ending up with redundant and
  ineffective thinking. This newly introduced scenario exacerbates the general
  overthinking issue to a large extent, which we name as the MiP-Overthinking.
  Such failures are against the ``test-time scaling law'' but have been widely
  observed on multiple datasets we curated with MiP, indicating the harm of cheap
  overthinking and a lack of critical thinking. Surprisingly, LLMs not
  specifically trained for reasoning exhibit much better performance on the MiP
  scenario, producing much shorter responses that quickly identify ill-posed
  queries. This implies a critical flaw of the current training recipe for
  reasoning LLMs, which does not encourage efficient thinking adequately, leading
  to the abuse of thinking patterns. To further investigate the reasons behind
  such failures, we conduct fine-grained analyses of the reasoning length,
  overthinking patterns, and location of critical thinking on different types of
  LLMs. Moreover, our extended ablation study reveals that the overthinking is
  contagious through the distillation of reasoning models' responses. These
  results improve the understanding of overthinking and shed novel insights into
  mitigating the problem.
paper_authors._id: 67f72e933eacf8888816f3b0
paper_authors.name: Chenrui Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:05:59.999Z
paper_authors._id: 64a8121e35fab7cd04c30ed0
paper_authors.avatarUrl: /avatars/48849b84703158772f1022932331b143.svg
paper_authors.fullname: Chenrui Fan
paper_authors.type: user
paper_authors.user: Fcr09
paper_authors._id: 67f72e933eacf8888816f3b1
paper_authors.name: Ming Li
paper_authors._id: 67f72e933eacf8888816f3b2
paper_authors.name: Lichao Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:06:12.092Z
paper_authors._id: 65a52766215aabac489e3468
paper_authors.avatarUrl: /avatars/fe05e22cd7e12e961296426434e17c76.svg
paper_authors.fullname: Lichao Sun
paper_authors.type: user
paper_authors.user: sunlichao137
paper_authors._id: 67f72e933eacf8888816f3b3
paper_authors.name: Tianyi Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-10T06:39:37.906Z
paper_authors._id: 647f5af5b0e96764589f3b2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
paper_authors.fullname: Tianyi Zhou
paper_authors.type: user
paper_authors.user: zhoutianyi
paper_title: Missing Premise exacerbates Overthinking: Are Reasoning Models losing
    Critical Thinking Skill?
paper_summary: We find that the response length of reasoning LLMs, whether trained by
  reinforcement learning or supervised learning, drastically increases for
  ill-posed questions with missing premises (MiP), ending up with redundant and
  ineffective thinking. This newly introduced scenario exacerbates the general
  overthinking issue to a large extent, which we name as the MiP-Overthinking.
  Such failures are against the ``test-time scaling law'' but have been widely
  observed on multiple datasets we curated with MiP, indicating the harm of cheap
  overthinking and a lack of critical thinking. Surprisingly, LLMs not
  specifically trained for reasoning exhibit much better performance on the MiP
  scenario, producing much shorter responses that quickly identify ill-posed
  queries. This implies a critical flaw of the current training recipe for
  reasoning LLMs, which does not encourage efficient thinking adequately, leading
  to the abuse of thinking patterns. To further investigate the reasons behind
  such failures, we conduct fine-grained analyses of the reasoning length,
  overthinking patterns, and location of critical thinking on different types of
  LLMs. Moreover, our extended ablation study reveals that the overthinking is
  contagious through the distillation of reasoning models' responses. These
  results improve the understanding of overthinking and shed novel insights into
  mitigating the problem.
paper_ai_summary: Reasoning LLMs generate redundant responses to ill-posed questions, a phenomenon named MiP-Overthinking, which is not observed in non-reasoning LLMs and spreads through model distillation.
paper_ai_keywords: LLMs reinforcement learning supervised learning ill-posed questions missing premises MiP overthinking test-time scaling law critical thinking fine-grained analyses reasoning length overthinking patterns critical thinking location distillation model distillation