title: Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM
    Reasoning
summary: Reasoning large language models (LLMs) heavily rely on scaling test-time
  compute to perform complex reasoning tasks by generating extensive "thinking"
  chains. While demonstrating impressive results, this approach incurs
  significant computational costs and inference time. In this work, we challenge
  the assumption that long thinking chains results in better reasoning
  capabilities. We first demonstrate that shorter reasoning chains within
  individual questions are significantly more likely to yield correct answers -
  up to 34.5% more accurate than the longest chain sampled for the same question.
  Based on these results, we suggest short-m@k, a novel reasoning LLM inference
  method. Our method executes k independent generations in parallel and halts
  computation once the first m thinking processes are done. The final answer is
  chosen using majority voting among these m chains. Basic short-1@k demonstrates
  similar or even superior performance over standard majority voting in
  low-compute settings - using up to 40% fewer thinking tokens. short-3@k, while
  slightly less efficient than short-1@k, consistently surpasses majority voting
  across all compute budgets, while still being substantially faster (up to 33%
  wall time reduction). Inspired by our results, we finetune an LLM using short,
  long, and randomly selected reasoning chains. We then observe that training on
  the shorter ones leads to better performance. Our findings suggest rethinking
  current methods of test-time compute in reasoning LLMs, emphasizing that longer
  "thinking" does not necessarily translate to improved performance and can,
  counter-intuitively, lead to degraded results.
paper_authors._id: 68368b76b399c7d3af071167
paper_authors.name: Michael Hassid
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:16:46.857Z
paper_authors._id: 6547411a9295970f878aa52e
paper_authors.avatarUrl: /avatars/6e240f0add27bf1a6c04a9618eccdf83.svg
paper_authors.fullname: Michael Hassid
paper_authors.type: user
paper_authors.user: hassid
paper_authors._id: 68368b76b399c7d3af071168
paper_authors.name: Gabriel Synnaeve
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:16:53.157Z
paper_authors._id: 630eac7931970d1cd4fbacf2
paper_authors.avatarUrl: /avatars/b7ccbddfa745db854dc342be1327cd53.svg
paper_authors.fullname: Gabriel Synnaeve
paper_authors.type: user
paper_authors.user: gsynnaeve
paper_authors._id: 68368b76b399c7d3af071169
paper_authors.name: Yossi Adi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:16:59.326Z
paper_authors._id: 6481e135578646b5c2386728
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6481e135578646b5c2386728/SPva4iNw0pORiCXD45cx9.jpeg
paper_authors.fullname: Yossi Adi
paper_authors.type: user
paper_authors.user: adiyoss
paper_authors._id: 68368b76b399c7d3af07116a
paper_authors.name: Roy Schwartz
paper_title: Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM
    Reasoning
paper_summary: Reasoning large language models (LLMs) heavily rely on scaling test-time
  compute to perform complex reasoning tasks by generating extensive "thinking"
  chains. While demonstrating impressive results, this approach incurs
  significant computational costs and inference time. In this work, we challenge
  the assumption that long thinking chains results in better reasoning
  capabilities. We first demonstrate that shorter reasoning chains within
  individual questions are significantly more likely to yield correct answers -
  up to 34.5% more accurate than the longest chain sampled for the same question.
  Based on these results, we suggest short-m@k, a novel reasoning LLM inference
  method. Our method executes k independent generations in parallel and halts
  computation once the first m thinking processes are done. The final answer is
  chosen using majority voting among these m chains. Basic short-1@k demonstrates
  similar or even superior performance over standard majority voting in
  low-compute settings - using up to 40% fewer thinking tokens. short-3@k, while
  slightly less efficient than short-1@k, consistently surpasses majority voting
  across all compute budgets, while still being substantially faster (up to 33%
  wall time reduction). Inspired by our results, we finetune an LLM using short,
  long, and randomly selected reasoning chains. We then observe that training on
  the shorter ones leads to better performance. Our findings suggest rethinking
  current methods of test-time compute in reasoning LLMs, emphasizing that longer
  "thinking" does not necessarily translate to improved performance and can,
  counter-intuitively, lead to degraded results.
paper_ai_summary: Shorter reasoning chains in LLMs can achieve similar or better performance with reduced computational cost and inference time compared to longer chains.
paper_ai_keywords: reasoning large language models thinking chains majority voting compute budgets wall time reduction