Missing Premise exacerbates Overthinking: Are Reasoning Models losing
  Critical Thinking Skill?
We find that the response length of reasoning LLMs, whether trained by
reinforcement learning or supervised learning, drastically increases for
ill-posed questions with missing premises (MiP), ending up with redundant and
ineffective thinking. This newly introduced scenario exacerbates the general
overthinking issue to a large extent, which we name as the MiP-Overthinking.
Such failures are against the ``test-time scaling law'' but have been widely
observed on multiple datasets we curated with MiP, indicating the harm of cheap
overthinking and a lack of critical thinking. Surprisingly, LLMs not
specifically trained for reasoning exhibit much better performance on the MiP
scenario, producing much shorter responses that quickly identify ill-posed
queries. This implies a critical flaw of the current training recipe for
reasoning LLMs, which does not encourage efficient thinking adequately, leading
to the abuse of thinking patterns. To further investigate the reasons behind
such failures, we conduct fine-grained analyses of the reasoning length,
overthinking patterns, and location of critical thinking on different types of
LLMs. Moreover, our extended ablation study reveals that the overthinking is
contagious through the distillation of reasoning models' responses. These
results improve the understanding of overthinking and shed novel insights into
mitigating the problem.
67f72e933eacf8888816f3b0
Chenrui Fan
admin_assigned
2025-04-10T08:05:59.999Z
64a8121e35fab7cd04c30ed0
/avatars/48849b84703158772f1022932331b143.svg
Chenrui Fan
user
Fcr09
67f72e933eacf8888816f3b1
Ming Li
67f72e933eacf8888816f3b2
Lichao Sun
admin_assigned
2025-04-10T08:06:12.092Z
65a52766215aabac489e3468
/avatars/fe05e22cd7e12e961296426434e17c76.svg
Lichao Sun
user
sunlichao137
67f72e933eacf8888816f3b3
Tianyi Zhou
claimed_verified
2025-04-10T06:39:37.906Z
647f5af5b0e96764589f3b2a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
Tianyi Zhou
user
zhoutianyi
Missing Premise exacerbates Overthinking: Are Reasoning Models losing
  Critical Thinking Skill?
We find that the response length of reasoning LLMs, whether trained by
reinforcement learning or supervised learning, drastically increases for
ill-posed questions with missing premises (MiP), ending up with redundant and
ineffective thinking. This newly introduced scenario exacerbates the general
overthinking issue to a large extent, which we name as the MiP-Overthinking.
Such failures are against the ``test-time scaling law'' but have been widely
observed on multiple datasets we curated with MiP, indicating the harm of cheap
overthinking and a lack of critical thinking. Surprisingly, LLMs not
specifically trained for reasoning exhibit much better performance on the MiP
scenario, producing much shorter responses that quickly identify ill-posed
queries. This implies a critical flaw of the current training recipe for
reasoning LLMs, which does not encourage efficient thinking adequately, leading
to the abuse of thinking patterns. To further investigate the reasons behind
such failures, we conduct fine-grained analyses of the reasoning length,
overthinking patterns, and location of critical thinking on different types of
LLMs. Moreover, our extended ablation study reveals that the overthinking is
contagious through the distillation of reasoning models' responses. These
results improve the understanding of overthinking and shed novel insights into
mitigating the problem.
Reasoning LLMs generate redundant responses to ill-posed questions, a phenomenon named MiP-Overthinking, which is not observed in non-reasoning LLMs and spreads through model distillation.
LLMs reinforcement learning supervised learning ill-posed questions missing premises MiP overthinking test-time scaling law critical thinking fine-grained analyses reasoning length overthinking patterns critical thinking location distillation model distillation