title: Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic
    Confidence
summary: Test-Time Scaling (TTS) methods for enhancing Large Language Model (LLM)
  reasoning often incur substantial computational costs, primarily due to
  extensive reliance on external Process Reward Models (PRMs) or sampling methods
  like Best-of-N (BoN). This paper introduces Guided by Gut (GG), an efficient
  self-guided TTS framework that achieves PRM-level performance without costly
  external verifier models. Our method employs a lightweight tree search guided
  solely by intrinsic LLM signals, token-level confidence and step novelty. One
  critical innovation is improving the reliability of internal confidence
  estimates via a targeted reinforcement learning fine-tuning phase. Empirical
  evaluations on challenging mathematical reasoning benchmarks demonstrate that
  GG enables smaller models (e.g., 1.5B parameters) to achieve accuracy matching
  or surpassing significantly larger models (e.g., 32B-70B parameters), while
  reducing GPU memory usage by up to 10x. Compared to PRM-based methods, GG
  achieves comparable accuracy with 8x faster inference speeds and 4-5x lower
  memory usage. Additionally, GG reduces KV cache memory usage by approximately
  50% compared to the BoN strategy, facilitating more efficient and practical
  deployment of TTS techniques.
paper_authors._id: 6836abd6871c9d398c136ee0
paper_authors.name: Amirhosein Ghasemabadi
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-28T07:04:37.500Z
paper_authors._id: 6830c4be7f72826192827659
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0hTKkO_zytpy9W8_0s291.png
paper_authors.fullname: Amirhosein Ghasemabadi
paper_authors.type: user
paper_authors.user: AmirhoseinGH
paper_authors._id: 6836abd6871c9d398c136ee1
paper_authors.name: Keith G. Mills
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-28T06:48:14.969Z
paper_authors._id: 661c391720b47b0daddfcc5a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RwvDJRtlEEPch27xId9Cb.png
paper_authors.fullname: Keith G. Mills
paper_authors.type: user
paper_authors.user: kgmills
paper_authors._id: 6836abd6871c9d398c136ee2
paper_authors.name: Baochun Li
paper_authors._id: 6836abd6871c9d398c136ee3
paper_authors.name: Di Niu
paper_title: Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic
    Confidence
paper_summary: Test-Time Scaling (TTS) methods for enhancing Large Language Model (LLM)
  reasoning often incur substantial computational costs, primarily due to
  extensive reliance on external Process Reward Models (PRMs) or sampling methods
  like Best-of-N (BoN). This paper introduces Guided by Gut (GG), an efficient
  self-guided TTS framework that achieves PRM-level performance without costly
  external verifier models. Our method employs a lightweight tree search guided
  solely by intrinsic LLM signals, token-level confidence and step novelty. One
  critical innovation is improving the reliability of internal confidence
  estimates via a targeted reinforcement learning fine-tuning phase. Empirical
  evaluations on challenging mathematical reasoning benchmarks demonstrate that
  GG enables smaller models (e.g., 1.5B parameters) to achieve accuracy matching
  or surpassing significantly larger models (e.g., 32B-70B parameters), while
  reducing GPU memory usage by up to 10x. Compared to PRM-based methods, GG
  achieves comparable accuracy with 8x faster inference speeds and 4-5x lower
  memory usage. Additionally, GG reduces KV cache memory usage by approximately
  50% compared to the BoN strategy, facilitating more efficient and practical
  deployment of TTS techniques.
paper_ai_summary: The Guided by Gut (GG) framework enhances LLM reasoning efficiently using intrinsic signals and token-level confidence, outperforming PRM-based methods with faster inference and lower memory usage.
paper_ai_keywords: Test-Time Scaling (TTS) Large Language Model (LLM) Process Reward Models (PRMs) Best-of-N (BoN) tree search intrinsic LLM signals token-level confidence step novelty targeted reinforcement learning fine-tuning