Steering LLM Thinking with Budget Guidance
Recent deep-thinking large language models often reason extensively to
improve performance, but such lengthy reasoning is not always desirable, as it
incurs excessive inference costs with disproportionate performance gains.
Controlling reasoning length without sacrificing performance is therefore
important, but remains challenging, especially under tight thinking budgets. We
propose budget guidance, a simple yet effective method for steering the
reasoning process of LLMs toward a target budget without requiring any LLM
fine-tuning. Our approach introduces a lightweight predictor that models a
Gamma distribution over the remaining thinking length during next-token
generation. This signal is then used to guide generation in a soft, token-level
manner, ensuring that the overall reasoning trace adheres to the specified
thinking budget. Budget guidance enables natural control of the thinking
length, along with significant token efficiency improvements over baseline
methods on challenging math benchmarks. For instance, it achieves up to a 26%
accuracy gain on the MATH-500 benchmark under tight budgets compared to
baseline methods, while maintaining competitive accuracy with only 63% of the
thinking tokens used by the full-thinking model. Budget guidance also
generalizes to broader task domains and exhibits emergent capabilities, such as
estimating question difficulty. The source code is available at:
https://github.com/UMass-Embodied-AGI/BudgetGuidance.
6850e1645e07650ecce890fa
Junyan Li
6850e1645e07650ecce890fb
Wenshuo Zhao
6850e1645e07650ecce890fc
Yang Zhang
6850e1645e07650ecce890fd
Chuang Gan
Steering LLM Thinking with Budget Guidance
Recent deep-thinking large language models often reason extensively to
improve performance, but such lengthy reasoning is not always desirable, as it
incurs excessive inference costs with disproportionate performance gains.
Controlling reasoning length without sacrificing performance is therefore
important, but remains challenging, especially under tight thinking budgets. We
propose budget guidance, a simple yet effective method for steering the
reasoning process of LLMs toward a target budget without requiring any LLM
fine-tuning. Our approach introduces a lightweight predictor that models a
Gamma distribution over the remaining thinking length during next-token
generation. This signal is then used to guide generation in a soft, token-level
manner, ensuring that the overall reasoning trace adheres to the specified
thinking budget. Budget guidance enables natural control of the thinking
length, along with significant token efficiency improvements over baseline
methods on challenging math benchmarks. For instance, it achieves up to a 26%
accuracy gain on the MATH-500 benchmark under tight budgets compared to
baseline methods, while maintaining competitive accuracy with only 63% of the
thinking tokens used by the full-thinking model. Budget guidance also
generalizes to broader task domains and exhibits emergent capabilities, such as
estimating question difficulty. The source code is available at:
https://github.com/UMass-Embodied-AGI/BudgetGuidance.
Budget guidance is a method that steers LLM reasoning within a targeted budget without fine-tuning and achieves improved efficiency and performance on math benchmarks.
deep-thinking large language models next-token generation Gamma distribution budget guidance thinking budget token efficiency natural control question difficulty estimation