title: Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
summary: The remarkable performance of models like the OpenAI o1 can be attributed to
  their ability to emulate human-like long-time thinking during inference. These
  models employ extended chain-of-thought (CoT) processes, exploring multiple
  strategies to enhance problem-solving capabilities. However, a critical
  question remains: How to intelligently and efficiently scale computational
  resources during testing. This paper presents the first comprehensive study on
  the prevalent issue of overthinking in these models, where excessive
  computational resources are allocated for simple problems with minimal benefit.
  We introduce novel efficiency metrics from both outcome and process
  perspectives to evaluate the rational use of computational resources by o1-like
  models. Using a self-training paradigm, we propose strategies to mitigate
  overthinking, streamlining reasoning processes without compromising accuracy.
  Experimental results show that our approach successfully reduces computational
  overhead while preserving model performance across a range of testsets with
  varying difficulty levels, such as GSM8K, MATH500, GPQA, and AIME.
paper_authors._id: 6773f75a23a7829936cb36bd
paper_authors.name: Xingyu Chen
paper_authors._id: 6773f75a23a7829936cb36be
paper_authors.name: Jiahao Xu
paper_authors._id: 6773f75a23a7829936cb36bf
paper_authors.name: Tian Liang
paper_authors._id: 6773f75a23a7829936cb36c0
paper_authors.name: Zhiwei He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-01T20:12:33.739Z
paper_authors._id: 638439ca834d3558a398d035
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669609868550-noauth.png
paper_authors.fullname: Zhiwei He
paper_authors.type: user
paper_authors.user: zwhe99
paper_authors._id: 6773f75a23a7829936cb36c1
paper_authors.name: Jianhui Pang
paper_authors._id: 6773f75a23a7829936cb36c2
paper_authors.name: Dian Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T20:46:22.967Z
paper_authors._id: 62d58fd53bf5e059f7cc3245
paper_authors.avatarUrl: /avatars/7a4f3ee4a37245f67efd26749d66a706.svg
paper_authors.fullname: Dian Yu
paper_authors.type: user
paper_authors.user: yudian
paper_authors._id: 6773f75a23a7829936cb36c3
paper_authors.name: Linfeng Song
paper_authors._id: 6773f75a23a7829936cb36c4
paper_authors.name: Qiuzhi Liu
paper_authors._id: 6773f75a23a7829936cb36c5
paper_authors.name: Mengfei Zhou
paper_authors._id: 6773f75a23a7829936cb36c6
paper_authors.name: Zhuosheng Zhang
paper_authors._id: 6773f75a23a7829936cb36c7
paper_authors.name: Rui Wang
paper_authors._id: 6773f75a23a7829936cb36c8
paper_authors.name: Zhaopeng Tu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T07:23:20.674Z
paper_authors._id: 67485743561b1e6f9579389f
paper_authors.avatarUrl: /avatars/8a4cc63bd7be388010bc329bb74582a1.svg
paper_authors.fullname: Zhaopeng Tu
paper_authors.type: user
paper_authors.user: zptu
paper_authors._id: 6773f75a23a7829936cb36c9
paper_authors.name: Haitao Mi
paper_authors._id: 6773f75a23a7829936cb36ca
paper_authors.name: Dong Yu
paper_title: Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
paper_summary: The remarkable performance of models like the OpenAI o1 can be attributed to
  their ability to emulate human-like long-time thinking during inference. These
  models employ extended chain-of-thought (CoT) processes, exploring multiple
  strategies to enhance problem-solving capabilities. However, a critical
  question remains: How to intelligently and efficiently scale computational
  resources during testing. This paper presents the first comprehensive study on
  the prevalent issue of overthinking in these models, where excessive
  computational resources are allocated for simple problems with minimal benefit.
  We introduce novel efficiency metrics from both outcome and process
  perspectives to evaluate the rational use of computational resources by o1-like
  models. Using a self-training paradigm, we propose strategies to mitigate
  overthinking, streamlining reasoning processes without compromising accuracy.
  Experimental results show that our approach successfully reduces computational
  overhead while preserving model performance across a range of testsets with
  varying difficulty levels, such as GSM8K, MATH500, GPQA, and AIME.
paper_ai_summary: The paper addresses overthinking in models like OpenAI o1 by introducing efficiency metrics and strategies to reduce computational overhead without sacrificing accuracy.
paper_ai_keywords: chain-of-thought overthinking self-training paradigm computational overhead accuracy