Token-Budget-Aware LLM Reasoning
Reasoning is critical for large language models (LLMs) to excel in a wide
range of tasks. While methods like Chain-of-Thought (CoT) reasoning enhance LLM
performance by decomposing problems into intermediate steps, they also incur
significant overhead in token usage, leading to increased costs. We find that
the reasoning process of current LLMs is unnecessarily lengthy and it can be
compressed by including a reasonable token budget in the prompt, but the choice
of token budget plays a crucial role in the actual compression effectiveness.
We then propose a token-budget-aware LLM reasoning framework, which dynamically
estimates token budgets for different problems based on reasoning complexity
and uses the estimated token budgets to guide the reasoning process.
Experiments show that our method effectively reduces token costs in CoT
reasoning with only a slight performance reduction, offering a practical
solution to balance efficiency and accuracy in LLM reasoning. Code:
https://github.com/GeniusHTX/TALE.
676c40d619a21c8b928d13c2
Tingxu Han
676c40d619a21c8b928d13c3
Chunrong Fang
676c40d619a21c8b928d13c4
Shiyu Zhao
676c40d619a21c8b928d13c5
Shiqing Ma
676c40d619a21c8b928d13c6
Zhenyu Chen
676c40d619a21c8b928d13c7
Zhenting Wang
claimed_verified
2024-12-30T19:32:16.437Z
64dfcc62e8b6f3f3baa950e0
/avatars/21bbff67d46c08044efe2406575aa77e.svg
Zhenting Wang
user
ztwang
Token-Budget-Aware LLM Reasoning
Reasoning is critical for large language models (LLMs) to excel in a wide
range of tasks. While methods like Chain-of-Thought (CoT) reasoning enhance LLM
performance by decomposing problems into intermediate steps, they also incur
significant overhead in token usage, leading to increased costs. We find that
the reasoning process of current LLMs is unnecessarily lengthy and it can be
compressed by including a reasonable token budget in the prompt, but the choice
of token budget plays a crucial role in the actual compression effectiveness.
We then propose a token-budget-aware LLM reasoning framework, which dynamically
estimates token budgets for different problems based on reasoning complexity
and uses the estimated token budgets to guide the reasoning process.
Experiments show that our method effectively reduces token costs in CoT
reasoning with only a slight performance reduction, offering a practical
solution to balance efficiency and accuracy in LLM reasoning. Code:
https://github.com/GeniusHTX/TALE.
A token-budget-aware framework dynamically estimates and allocates token budgets for LLM reasoning, reducing costs with minimal performance loss.
Chain-of-Thought CoT reasoning token budget token usage reasoning complexity