title: Token-Budget-Aware LLM Reasoning
summary: Reasoning is critical for large language models (LLMs) to excel in a wide
  range of tasks. While methods like Chain-of-Thought (CoT) reasoning enhance LLM
  performance by decomposing problems into intermediate steps, they also incur
  significant overhead in token usage, leading to increased costs. We find that
  the reasoning process of current LLMs is unnecessarily lengthy and it can be
  compressed by including a reasonable token budget in the prompt, but the choice
  of token budget plays a crucial role in the actual compression effectiveness.
  We then propose a token-budget-aware LLM reasoning framework, which dynamically
  estimates token budgets for different problems based on reasoning complexity
  and uses the estimated token budgets to guide the reasoning process.
  Experiments show that our method effectively reduces token costs in CoT
  reasoning with only a slight performance reduction, offering a practical
  solution to balance efficiency and accuracy in LLM reasoning. Code:
  https://github.com/GeniusHTX/TALE.
paper_authors._id: 676c40d619a21c8b928d13c2
paper_authors.name: Tingxu Han
paper_authors._id: 676c40d619a21c8b928d13c3
paper_authors.name: Chunrong Fang
paper_authors._id: 676c40d619a21c8b928d13c4
paper_authors.name: Shiyu Zhao
paper_authors._id: 676c40d619a21c8b928d13c5
paper_authors.name: Shiqing Ma
paper_authors._id: 676c40d619a21c8b928d13c6
paper_authors.name: Zhenyu Chen
paper_authors._id: 676c40d619a21c8b928d13c7
paper_authors.name: Zhenting Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:32:16.437Z
paper_authors._id: 64dfcc62e8b6f3f3baa950e0
paper_authors.avatarUrl: /avatars/21bbff67d46c08044efe2406575aa77e.svg
paper_authors.fullname: Zhenting Wang
paper_authors.type: user
paper_authors.user: ztwang
paper_title: Token-Budget-Aware LLM Reasoning
paper_summary: Reasoning is critical for large language models (LLMs) to excel in a wide
  range of tasks. While methods like Chain-of-Thought (CoT) reasoning enhance LLM
  performance by decomposing problems into intermediate steps, they also incur
  significant overhead in token usage, leading to increased costs. We find that
  the reasoning process of current LLMs is unnecessarily lengthy and it can be
  compressed by including a reasonable token budget in the prompt, but the choice
  of token budget plays a crucial role in the actual compression effectiveness.
  We then propose a token-budget-aware LLM reasoning framework, which dynamically
  estimates token budgets for different problems based on reasoning complexity
  and uses the estimated token budgets to guide the reasoning process.
  Experiments show that our method effectively reduces token costs in CoT
  reasoning with only a slight performance reduction, offering a practical
  solution to balance efficiency and accuracy in LLM reasoning. Code:
  https://github.com/GeniusHTX/TALE.
paper_ai_keywords: Chain-of-Thought (CoT) reasoning token budget token usage LLM reasoning framework feed-wise reasoning performance reduction token costs efficiency and accuracy