title: Revisiting the Test-Time Scaling of o1-like Models: Do they Truly
    Possess Test-Time Scaling Capabilities?
summary: The advent of test-time scaling in large language models (LLMs), exemplified
  by OpenAI's o1 series, has advanced reasoning capabilities by scaling
  computational resource allocation during inference. While successors like QwQ,
  Deepseek-R1 (R1) and LIMO replicate these advancements, whether these models
  truly possess test-time scaling capabilities remains underexplored. This study
  found that longer CoTs of these o1-like models do not consistently enhance
  accuracy; in fact, correct solutions are often shorter than incorrect ones for
  the same questions. Further investigation shows this phenomenon is closely
  related to models' self-revision capabilities - longer CoTs contain more
  self-revisions, which often lead to performance degradation. We then compare
  sequential and parallel scaling strategies on QwQ, R1 and LIMO, finding that
  parallel scaling achieves better coverage and scalability. Based on these
  insights, we propose Shortest Majority Vote, a method that combines parallel
  scaling strategies with CoT length characteristics, significantly improving
  models' test-time scalability compared to conventional majority voting
  approaches.
paper_authors._id: 67b56007fa141a55e51d9d78
paper_authors.name: Zhiyuan Zeng
paper_authors._id: 67b56007fa141a55e51d9d79
paper_authors.name: Qinyuan Cheng
paper_authors._id: 67b56007fa141a55e51d9d7a
paper_authors.name: Zhangyue Yin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T11:50:17.993Z
paper_authors._id: 628c5da32f09ccf530204dbe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1653366416287-628c5da32f09ccf530204dbe.jpeg
paper_authors.fullname: Zhangyue Yin
paper_authors.type: user
paper_authors.user: yinzhangyue
paper_authors._id: 67b56007fa141a55e51d9d7b
paper_authors.name: Yunhua Zhou
paper_authors._id: 67b56007fa141a55e51d9d7c
paper_authors.name: Xipeng Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T11:50:29.919Z
paper_authors._id: 61457b8deff2c9fdb4de4988
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1632381702899-61457b8deff2c9fdb4de4988.jpeg
paper_authors.fullname: Xipeng Qiu
paper_authors.type: user
paper_authors.user: xpqiu
paper_title: Revisiting the Test-Time Scaling of o1-like Models: Do they Truly
    Possess Test-Time Scaling Capabilities?
paper_summary: The advent of test-time scaling in large language models (LLMs), exemplified
  by OpenAI's o1 series, has advanced reasoning capabilities by scaling
  computational resource allocation during inference. While successors like QwQ,
  Deepseek-R1 (R1) and LIMO replicate these advancements, whether these models
  truly possess test-time scaling capabilities remains underexplored. This study
  found that longer CoTs of these o1-like models do not consistently enhance
  accuracy; in fact, correct solutions are often shorter than incorrect ones for
  the same questions. Further investigation shows this phenomenon is closely
  related to models' self-revision capabilities - longer CoTs contain more
  self-revisions, which often lead to performance degradation. We then compare
  sequential and parallel scaling strategies on QwQ, R1 and LIMO, finding that
  parallel scaling achieves better coverage and scalability. Based on these
  insights, we propose Shortest Majority Vote, a method that combines parallel
  scaling strategies with CoT length characteristics, significantly improving
  models' test-time scalability compared to conventional majority voting
  approaches.
paper_ai_summary: Shortest Majority Vote enhances test-time scalability in large language models by leveraging parallel scaling and optimal CoT lengths.
paper_ai_keywords: test-time scaling large language models (LLMs) O1 series QwQ Deepseek-R1 (R1) LIMO CoTs self-revision capabilities sequential scaling parallel scaling