title: When To Solve, When To Verify: Compute-Optimal Problem Solving and
    Generative Verification for LLM Reasoning
summary: Scaling test-time compute has emerged as a key strategy for enhancing the
  reasoning capabilities of large language models (LLMs), particularly in tasks
  like mathematical problem-solving. A traditional approach, Self-Consistency
  (SC), generates multiple solutions to a problem and selects the most common
  answer via majority voting. Another common method involves scoring each
  solution with a reward model (verifier) and choosing the best one. Recent
  advancements in Generative Reward Models (GenRM) reframe verification as a
  next-token prediction task, enabling inference-time scaling along a new axis.
  Specifically, GenRM generates multiple verification chains-of-thought to score
  each solution. Under a limited inference budget, this introduces a fundamental
  trade-off: should you spend the budget on scaling solutions via SC or generate
  fewer solutions and allocate compute to verification via GenRM? To address
  this, we evaluate GenRM against SC under a fixed inference budget.
  Interestingly, we find that SC is more compute-efficient than GenRM for most
  practical inference budgets across diverse models and datasets. For instance,
  GenRM first matches SC after consuming up to 8x the inference compute and
  requires significantly more compute to outperform it. Furthermore, we derive
  inference scaling laws for the GenRM paradigm, revealing that compute-optimal
  inference favors scaling solution generation more aggressively than scaling the
  number of verifications. Our work provides practical guidance on optimizing
  test-time scaling by balancing solution generation and verification. The code
  is available at https://github.com/nishadsinghi/sc-genrm-scaling.
paper_authors._id: 67ec95ea3d267d26663ea34b
paper_authors.name: Nishad Singhi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T13:56:07.949Z
paper_authors._id: 641f72d8eefe94aff6e21068
paper_authors.avatarUrl: /avatars/0f07cffb49698232c4bd912898abc920.svg
paper_authors.fullname: Nishad Singhi
paper_authors.type: user
paper_authors.user: nishadsinghi
paper_authors._id: 67ec95ea3d267d26663ea34c
paper_authors.name: Hritik Bansal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T11:16:19.164Z
paper_authors._id: 61c5c25705aa54027c52f7b3
paper_authors.avatarUrl: /avatars/8a89e040dc331b7a83d9a704c4fc29d2.svg
paper_authors.fullname: Hritik Bansal
paper_authors.type: user
paper_authors.user: hbXNov
paper_authors._id: 67ec95ea3d267d26663ea34d
paper_authors.name: Arian Hosseini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T11:17:10.400Z
paper_authors._id: 62e447e9640206c5d21e5ff6
paper_authors.avatarUrl: /avatars/42d8247c2ffa98a4c6255ce84051f1b9.svg
paper_authors.fullname: Arian Hosseini
paper_authors.type: user
paper_authors.user: arianhosseini
paper_authors._id: 67ec95ea3d267d26663ea34e
paper_authors.name: Aditya Grover
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T11:17:22.994Z
paper_authors._id: 65d38c1e5d5749637491679b
paper_authors.avatarUrl: /avatars/010a6588f382d6d4394216549a2385e3.svg
paper_authors.fullname: Aditya Grover
paper_authors.type: user
paper_authors.user: adityagrover
paper_authors._id: 67ec95ea3d267d26663ea34f
paper_authors.name: Kai-Wei Chang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T11:17:29.569Z
paper_authors._id: 60b7b9d71b90c5d07c23fbd0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1622653364258-noauth.jpeg
paper_authors.fullname: Kai-Wei Chang
paper_authors.type: user
paper_authors.user: kaiweichang
paper_authors._id: 67ec95ea3d267d26663ea350
paper_authors.name: Marcus Rohrbach
paper_authors._id: 67ec95ea3d267d26663ea351
paper_authors.name: Anna Rohrbach
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T11:16:35.830Z
paper_authors._id: 6397ca710bdcc7e3216f3c49
paper_authors.avatarUrl: /avatars/51ad23a000eaf817c2e47149810b0156.svg
paper_authors.fullname: Anna Rohrbach
paper_authors.type: user
paper_authors.user: anna-rohrbach
paper_title: When To Solve, When To Verify: Compute-Optimal Problem Solving and
    Generative Verification for LLM Reasoning
paper_summary: Scaling test-time compute has emerged as a key strategy for enhancing the
  reasoning capabilities of large language models (LLMs), particularly in tasks
  like mathematical problem-solving. A traditional approach, Self-Consistency
  (SC), generates multiple solutions to a problem and selects the most common
  answer via majority voting. Another common method involves scoring each
  solution with a reward model (verifier) and choosing the best one. Recent
  advancements in Generative Reward Models (GenRM) reframe verification as a
  next-token prediction task, enabling inference-time scaling along a new axis.
  Specifically, GenRM generates multiple verification chains-of-thought to score
  each solution. Under a limited inference budget, this introduces a fundamental
  trade-off: should you spend the budget on scaling solutions via SC or generate
  fewer solutions and allocate compute to verification via GenRM? To address
  this, we evaluate GenRM against SC under a fixed inference budget.
  Interestingly, we find that SC is more compute-efficient than GenRM for most
  practical inference budgets across diverse models and datasets. For instance,
  GenRM first matches SC after consuming up to 8x the inference compute and
  requires significantly more compute to outperform it. Furthermore, we derive
  inference scaling laws for the GenRM paradigm, revealing that compute-optimal
  inference favors scaling solution generation more aggressively than scaling the
  number of verifications. Our work provides practical guidance on optimizing
  test-time scaling by balancing solution generation and verification. The code
  is available at https://github.com/nishadsinghi/sc-genrm-scaling.
paper_ai_summary: Self-Consistency is more compute-efficient than Generative Reward Models for most practical inference budgets in large language models.
paper_ai_keywords: Self-Consistency Generative Reward Models next-token prediction chains-of-thought inference scaling laws