title: φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time
    Exploration and Exploitation
summary: Inference-time optimization scales computation to derive deliberate reasoning
  steps for effective performance. While previous search-based strategies address
  the short-sightedness of auto-regressive generation, the vast search space
  leads to excessive exploration and insufficient exploitation. To strike an
  efficient balance to derive the optimal step, we frame the decoding strategy as
  foresight sampling, leveraging simulated future steps to obtain globally
  optimal step estimation. Built on it, we propose a novel decoding strategy,
  named phi-Decoding. To provide a precise and expressive estimation of step
  value, phi-Decoding approximates two distributions via foresight and
  clustering. Sampling from the joint distribution, the optimal steps can be
  selected for exploitation. To support adaptive computation allocation, we
  propose in-width and in-depth pruning strategies, featuring a light-weight
  solution to achieve inference efficiency. Extensive experiments across seven
  benchmarks show phi-Decoding outperforms strong baselines in both
  performance and efficiency. Additional analysis demonstrates its generalization
  across various LLMs and scalability across a wide range of computing budgets.
  The code will be released at https://github.com/xufangzhi/phi-Decoding, and the
  open-source PyPI package is coming soon.
paper_authors._id: 67dbc49d85eacb364e913c38
paper_authors.name: Fangzhi Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T10:44:57.855Z
paper_authors._id: 64e6cf78ecce34cb442dc889
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
paper_authors.fullname: Fangzhi Xu
paper_authors.type: user
paper_authors.user: xufangzhi
paper_authors._id: 67dbc49d85eacb364e913c39
paper_authors.name: Hang Yan
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-20T09:48:32.179Z
paper_authors._id: 67dbe3d969655e406fda64b8
paper_authors.avatarUrl: /avatars/6053c84e32d0e46dd1e490c493f766ed.svg
paper_authors.fullname: Mei Tuan
paper_authors.type: user
paper_authors.user: Meituannnnnn
paper_authors._id: 67dbc49d85eacb364e913c3a
paper_authors.name: Chang Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T11:05:18.661Z
paper_authors._id: 637f22fd932a61b89aeeea37
paper_authors.avatarUrl: /avatars/342957f8242d4edaf1d58e1274313afe.svg
paper_authors.fullname: Chang Ma
paper_authors.type: user
paper_authors.user: changma
paper_authors._id: 67dbc49d85eacb364e913c3b
paper_authors.name: Haiteng Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T11:05:24.460Z
paper_authors._id: 64a7c6e223622f7f189bcbe1
paper_authors.avatarUrl: /avatars/4f13a7ed0d2b8d8dfff7dc650e46450a.svg
paper_authors.fullname: haiteng zhao
paper_authors.type: user
paper_authors.user: haitengzhao
paper_authors._id: 67dbc49d85eacb364e913c3c
paper_authors.name: Jun Liu
paper_authors._id: 67dbc49d85eacb364e913c3d
paper_authors.name: Qika Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T11:05:34.255Z
paper_authors._id: 66ac77011cfb12c087605acb
paper_authors.avatarUrl: /avatars/54c06bd1c4c9d491470ed4162c2301ae.svg
paper_authors.fullname: Lin
paper_authors.type: user
paper_authors.user: Qika
paper_authors._id: 67dbc49d85eacb364e913c3e
paper_authors.name: Zhiyong Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-20T11:05:41.288Z
paper_authors._id: 6280e830e99dccaac4bbfde5
paper_authors.avatarUrl: /avatars/9242b8d2826ce2f79af9bb794bba2b61.svg
paper_authors.fullname: Zhiyong Wu
paper_authors.type: user
paper_authors.user: zy001
paper_title: φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time
    Exploration and Exploitation
paper_summary: Inference-time optimization scales computation to derive deliberate reasoning
  steps for effective performance. While previous search-based strategies address
  the short-sightedness of auto-regressive generation, the vast search space
  leads to excessive exploration and insufficient exploitation. To strike an
  efficient balance to derive the optimal step, we frame the decoding strategy as
  foresight sampling, leveraging simulated future steps to obtain globally
  optimal step estimation. Built on it, we propose a novel decoding strategy,
  named phi-Decoding. To provide a precise and expressive estimation of step
  value, phi-Decoding approximates two distributions via foresight and
  clustering. Sampling from the joint distribution, the optimal steps can be
  selected for exploitation. To support adaptive computation allocation, we
  propose in-width and in-depth pruning strategies, featuring a light-weight
  solution to achieve inference efficiency. Extensive experiments across seven
  benchmarks show phi-Decoding outperforms strong baselines in both
  performance and efficiency. Additional analysis demonstrates its generalization
  across various LLMs and scalability across a wide range of computing budgets.
  The code will be released at https://github.com/xufangzhi/phi-Decoding, and the
  open-source PyPI package is coming soon.
paper_ai_summary: A new decoding strategy, $\phi$-Decoding, uses foresight sampling and pruning techniques to enhance the efficiency and performance of large language models during inference.
paper_ai_keywords: foresight sampling $\phi$-Decoding decoding strategy step estimation simulated future steps optimal step adaptive computation allocation in-width pruning in-depth pruning LLMs scalability computing budgets