title: Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time
    Scaling
summary: Test-Time Scaling (TTS) is an important method for improving the performance
  of Large Language Models (LLMs) by using additional computation during the
  inference phase. However, current studies do not systematically analyze how
  policy models, Process Reward Models (PRMs), and problem difficulty influence
  TTS. This lack of analysis limits the understanding and practical use of TTS
  methods. In this paper, we focus on two core questions: (1) What is the optimal
  approach to scale test-time computation across different policy models, PRMs,
  and problem difficulty levels? (2) To what extent can extended computation
  improve the performance of LLMs on complex tasks, and can smaller language
  models outperform larger ones through this approach? Through comprehensive
  experiments on MATH-500 and challenging AIME24 tasks, we have the following
  observations: (1) The compute-optimal TTS strategy is highly dependent on the
  choice of policy model, PRM, and problem difficulty. (2) With our
  compute-optimal TTS strategy, extremely small policy models can outperform
  larger models. For example, a 1B LLM can exceed a 405B LLM on MATH-500.
  Moreover, on both MATH-500 and AIME24, a 0.5B LLM outperforms GPT-4o, a 3B LLM
  surpasses a 405B LLM, and a 7B LLM beats o1 and DeepSeek-R1, while with higher
  inference efficiency. These findings show the significance of adapting TTS
  strategies to the specific characteristics of each task and model and indicate
  that TTS is a promising approach for enhancing the reasoning abilities of LLMs.
paper_authors._id: 67aabf93c0f8648f68c68ce4
paper_authors.name: Runze Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T07:55:22.940Z
paper_authors._id: 667187ba9ab144eb3ac43a1b
paper_authors.avatarUrl: /avatars/db5558aa1c5160b9aee8b58573271959.svg
paper_authors.fullname: Runze Liu
paper_authors.type: user
paper_authors.user: RyanLiu112
paper_authors._id: 67aabf93c0f8648f68c68ce5
paper_authors.name: Junqi Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-11T15:54:46.128Z
paper_authors._id: 67ab05fe4c6ca2d5db4c0c52
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/QpGUNDkeuKjX71s2GXlXF.png
paper_authors.fullname: Junqi Gao
paper_authors.type: user
paper_authors.user: ChetKao
paper_authors._id: 67aabf93c0f8648f68c68ce6
paper_authors.name: Jian Zhao
paper_authors._id: 67aabf93c0f8648f68c68ce7
paper_authors.name: Kaiyan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T07:55:18.725Z
paper_authors._id: 60bc94cd85a3ab33829b6211
paper_authors.avatarUrl: /avatars/b57d36c7577fbbb42ea5b963eef4144a.svg
paper_authors.fullname: Kaiyan Zhang
paper_authors.type: user
paper_authors.user: iseesaw
paper_authors._id: 67aabf93c0f8648f68c68ce8
paper_authors.name: Xiu Li
paper_authors._id: 67aabf93c0f8648f68c68ce9
paper_authors.name: Biqing Qi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-11T15:55:23.328Z
paper_authors._id: 645d9c3058f9ee315148116d
paper_authors.avatarUrl: /avatars/165e18f27b5a50738bf1d22857118478.svg
paper_authors.fullname: Biqing Qi
paper_authors.type: user
paper_authors.user: jackqi7
paper_authors._id: 67aabf93c0f8648f68c68cea
paper_authors.name: Wanli Ouyang
paper_authors._id: 67aabf93c0f8648f68c68ceb
paper_authors.name: Bowen Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-11T15:55:11.315Z
paper_authors._id: 669f614b59adf5b56e05bce3
paper_authors.avatarUrl: /avatars/ffd4189efbceb0e63a03db273065a44b.svg
paper_authors.fullname: BowenZhou
paper_authors.type: user
paper_authors.user: bowenZhou
paper_title: Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time
    Scaling
paper_summary: Test-Time Scaling (TTS) is an important method for improving the performance
  of Large Language Models (LLMs) by using additional computation during the
  inference phase. However, current studies do not systematically analyze how
  policy models, Process Reward Models (PRMs), and problem difficulty influence
  TTS. This lack of analysis limits the understanding and practical use of TTS
  methods. In this paper, we focus on two core questions: (1) What is the optimal
  approach to scale test-time computation across different policy models, PRMs,
  and problem difficulty levels? (2) To what extent can extended computation
  improve the performance of LLMs on complex tasks, and can smaller language
  models outperform larger ones through this approach? Through comprehensive
  experiments on MATH-500 and challenging AIME24 tasks, we have the following
  observations: (1) The compute-optimal TTS strategy is highly dependent on the
  choice of policy model, PRM, and problem difficulty. (2) With our
  compute-optimal TTS strategy, extremely small policy models can outperform
  larger models. For example, a 1B LLM can exceed a 405B LLM on MATH-500.
  Moreover, on both MATH-500 and AIME24, a 0.5B LLM outperforms GPT-4o, a 3B LLM
  surpasses a 405B LLM, and a 7B LLM beats o1 and DeepSeek-R1, while with higher
  inference efficiency. These findings show the significance of adapting TTS
  strategies to the specific characteristics of each task and model and indicate
  that TTS is a promising approach for enhancing the reasoning abilities of LLMs.
paper_ai_keywords: Test-Time Scaling (TTS) Large Language Models (LLMs) Policy models Process Reward Models (PRMs) MATH-500 AIME24 compute-optimal TTS strategy inference efficiency