title: LIMOPro: Reasoning Refinement for Efficient and Effective Test-time
    Scaling
summary: Large language models (LLMs) have demonstrated remarkable reasoning
  capabilities through test-time scaling approaches, particularly when fine-tuned
  with chain-of-thought (CoT) data distilled from more powerful large reasoning
  models (LRMs). However, these reasoning chains often contain verbose elements
  that mirror human problem-solving, categorized as progressive reasoning (the
  essential solution development path) and functional elements (verification
  processes, alternative solution approaches, and error corrections). While
  progressive reasoning is crucial, the functional elements significantly
  increase computational demands during test-time inference. We introduce PIR
  (Perplexity-based Importance Refinement), a principled framework that
  quantitatively evaluates the importance of each reasoning step based on its
  impact on answer prediction confidence. PIR systematically identifies and
  selectively prunes only low-importance functional steps while preserving
  progressive reasoning components, creating optimized training data that
  maintains the integrity of the core solution path while reducing verbosity.
  Models fine-tuned on PIR-optimized data exhibit superior test-time scaling
  properties, generating more concise reasoning chains while achieving improved
  accuracy (+0.9\% to +6.6\%) with significantly reduced token usage (-3\% to
  -41\%) across challenging reasoning benchmarks (AIME, AMC, and GPQA Diamond).
  Our approach demonstrates strong generalizability across different model sizes,
  data sources, and token budgets, offering a practical solution for deploying
  reasoning-capable LLMs in scenarios where efficient test-time scaling, response
  time, and computational efficiency are valuable constraints.
paper_authors._id: 6837210455e9bab4e9c302b1
paper_authors.name: Yang Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T19:13:52.090Z
paper_authors._id: 6002c316698168af3bb9f4a6
paper_authors.avatarUrl: /avatars/1028860b59951be04e94b126b0985dc0.svg
paper_authors.fullname: yangxiao
paper_authors.type: user
paper_authors.user: YangXiao-nlp
paper_authors._id: 6837210455e9bab4e9c302b2
paper_authors.name: Jiashuo Wang
paper_authors._id: 6837210455e9bab4e9c302b3
paper_authors.name: Ruifeng Yuan
paper_authors._id: 6837210455e9bab4e9c302b4
paper_authors.name: Chunpu Xu
paper_authors._id: 6837210455e9bab4e9c302b5
paper_authors.name: Kaishuai Xu
paper_authors._id: 6837210455e9bab4e9c302b6
paper_authors.name: Wenjie Li
paper_authors._id: 6837210455e9bab4e9c302b7
paper_authors.name: Pengfei Liu
paper_title: LIMOPro: Reasoning Refinement for Efficient and Effective Test-time
    Scaling
paper_summary: Large language models (LLMs) have demonstrated remarkable reasoning
  capabilities through test-time scaling approaches, particularly when fine-tuned
  with chain-of-thought (CoT) data distilled from more powerful large reasoning
  models (LRMs). However, these reasoning chains often contain verbose elements
  that mirror human problem-solving, categorized as progressive reasoning (the
  essential solution development path) and functional elements (verification
  processes, alternative solution approaches, and error corrections). While
  progressive reasoning is crucial, the functional elements significantly
  increase computational demands during test-time inference. We introduce PIR
  (Perplexity-based Importance Refinement), a principled framework that
  quantitatively evaluates the importance of each reasoning step based on its
  impact on answer prediction confidence. PIR systematically identifies and
  selectively prunes only low-importance functional steps while preserving
  progressive reasoning components, creating optimized training data that
  maintains the integrity of the core solution path while reducing verbosity.
  Models fine-tuned on PIR-optimized data exhibit superior test-time scaling
  properties, generating more concise reasoning chains while achieving improved
  accuracy (+0.9\% to +6.6\%) with significantly reduced token usage (-3\% to
  -41\%) across challenging reasoning benchmarks (AIME, AMC, and GPQA Diamond).
  Our approach demonstrates strong generalizability across different model sizes,
  data sources, and token budgets, offering a practical solution for deploying
  reasoning-capable LLMs in scenarios where efficient test-time scaling, response
  time, and computational efficiency are valuable constraints.
paper_ai_summary: A framework called PIR refines the importance of reasoning steps in large language models by pruning low-importance functional elements, leading to more concise reasoning chains with improved accuracy and reduced computational demands.
paper_ai_keywords: large language models chain-of-thought large reasoning models progressive reasoning functional elements perplexity-based importance refinement token usage reasoning benchmarks AIME AMC GPQA Diamond