title: Value-Guided Search for Efficient Chain-of-Thought Reasoning
summary: In this paper, we propose a simple and efficient method for value model
  training on long-context reasoning traces. Compared to existing process reward
  models (PRMs), our method does not require a fine-grained notion of "step,"
  which is difficult to define for long-context reasoning models. By collecting a
  dataset of 2.5 million reasoning traces, we train a 1.5B token-level value
  model and apply it to DeepSeek models for improved performance with test-time
  compute scaling. We find that block-wise value-guided search (VGS) with a final
  weighted majority vote achieves better test-time scaling than standard methods
  such as majority voting or best-of-n. With an inference budget of 64
  generations, VGS with DeepSeek-R1-Distill-1.5B achieves an average accuracy of
  45.7% across four competition math benchmarks (AIME 2024 & 2025, HMMT Feb 2024
  & 2025), reaching parity with o3-mini-medium. Moreover, VGS significantly
  reduces the inference FLOPs required to achieve the same performance of
  majority voting. Our dataset, model and codebase are open-sourced.
paper_authors._id: 6834ab9e6b152a062eb05453
paper_authors.name: Kaiwen Wang
paper_authors._id: 6834ab9e6b152a062eb05454
paper_authors.name: Jin Peng Zhou
paper_authors._id: 6834ab9e6b152a062eb05455
paper_authors.name: Jonathan Chang
paper_authors._id: 6834ab9e6b152a062eb05456
paper_authors.name: Zhaolin Gao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:55:06.339Z
paper_authors._id: 652eec0aabc673c4204c459e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/652eec0aabc673c4204c459e/9otSQFP8G3S8zarR1Y5rE.jpeg
paper_authors.fullname: Zhaolin Gao
paper_authors.type: user
paper_authors.user: GitBag
paper_authors._id: 6834ab9e6b152a062eb05457
paper_authors.name: Nathan Kallus
paper_authors._id: 6834ab9e6b152a062eb05458
paper_authors.name: Kianté Brantley
paper_authors._id: 6834ab9e6b152a062eb05459
paper_authors.name: Wen Sun
paper_title: Value-Guided Search for Efficient Chain-of-Thought Reasoning
paper_summary: In this paper, we propose a simple and efficient method for value model
  training on long-context reasoning traces. Compared to existing process reward
  models (PRMs), our method does not require a fine-grained notion of "step,"
  which is difficult to define for long-context reasoning models. By collecting a
  dataset of 2.5 million reasoning traces, we train a 1.5B token-level value
  model and apply it to DeepSeek models for improved performance with test-time
  compute scaling. We find that block-wise value-guided search (VGS) with a final
  weighted majority vote achieves better test-time scaling than standard methods
  such as majority voting or best-of-n. With an inference budget of 64
  generations, VGS with DeepSeek-R1-Distill-1.5B achieves an average accuracy of
  45.7% across four competition math benchmarks (AIME 2024 & 2025, HMMT Feb 2024
  & 2025), reaching parity with o3-mini-medium. Moreover, VGS significantly
  reduces the inference FLOPs required to achieve the same performance of
  majority voting. Our dataset, model and codebase are open-sourced.
paper_ai_summary: A simple and efficient method for value model training on long-context reasoning traces improves test-time performance and reduces computational cost compared to existing methods.
paper_ai_keywords: value model training long-context reasoning process reward models token-level value model block-wise value-guided search VGS inference FLOPs