φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time
  Exploration and Exploitation
Inference-time optimization scales computation to derive deliberate reasoning
steps for effective performance. While previous search-based strategies address
the short-sightedness of auto-regressive generation, the vast search space
leads to excessive exploration and insufficient exploitation. To strike an
efficient balance to derive the optimal step, we frame the decoding strategy as
foresight sampling, leveraging simulated future steps to obtain globally
optimal step estimation. Built on it, we propose a novel decoding strategy,
named phi-Decoding. To provide a precise and expressive estimation of step
value, phi-Decoding approximates two distributions via foresight and
clustering. Sampling from the joint distribution, the optimal steps can be
selected for exploitation. To support adaptive computation allocation, we
propose in-width and in-depth pruning strategies, featuring a light-weight
solution to achieve inference efficiency. Extensive experiments across seven
benchmarks show phi-Decoding outperforms strong baselines in both
performance and efficiency. Additional analysis demonstrates its generalization
across various LLMs and scalability across a wide range of computing budgets.
The code will be released at https://github.com/xufangzhi/phi-Decoding, and the
open-source PyPI package is coming soon.
67dbc49d85eacb364e913c38
Fangzhi Xu
claimed_verified
2025-03-20T10:44:57.855Z
64e6cf78ecce34cb442dc889
https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
Fangzhi Xu
user
xufangzhi
67dbc49d85eacb364e913c39
Hang Yan
extracted_pending
2025-03-20T09:48:32.179Z
67dbe3d969655e406fda64b8
/avatars/6053c84e32d0e46dd1e490c493f766ed.svg
Mei Tuan
user
Meituannnnnn
67dbc49d85eacb364e913c3a
Chang Ma
admin_assigned
2025-03-20T11:05:18.661Z
637f22fd932a61b89aeeea37
/avatars/342957f8242d4edaf1d58e1274313afe.svg
Chang Ma
user
changma
67dbc49d85eacb364e913c3b
Haiteng Zhao
admin_assigned
2025-03-20T11:05:24.460Z
64a7c6e223622f7f189bcbe1
/avatars/4f13a7ed0d2b8d8dfff7dc650e46450a.svg
haiteng zhao
user
haitengzhao
67dbc49d85eacb364e913c3c
Jun Liu
67dbc49d85eacb364e913c3d
Qika Lin
admin_assigned
2025-03-20T11:05:34.255Z
66ac77011cfb12c087605acb
/avatars/54c06bd1c4c9d491470ed4162c2301ae.svg
Lin
user
Qika
67dbc49d85eacb364e913c3e
Zhiyong Wu
admin_assigned
2025-03-20T11:05:41.288Z
6280e830e99dccaac4bbfde5
/avatars/9242b8d2826ce2f79af9bb794bba2b61.svg
Zhiyong Wu
user
zy001
φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time
  Exploration and Exploitation
Inference-time optimization scales computation to derive deliberate reasoning
steps for effective performance. While previous search-based strategies address
the short-sightedness of auto-regressive generation, the vast search space
leads to excessive exploration and insufficient exploitation. To strike an
efficient balance to derive the optimal step, we frame the decoding strategy as
foresight sampling, leveraging simulated future steps to obtain globally
optimal step estimation. Built on it, we propose a novel decoding strategy,
named phi-Decoding. To provide a precise and expressive estimation of step
value, phi-Decoding approximates two distributions via foresight and
clustering. Sampling from the joint distribution, the optimal steps can be
selected for exploitation. To support adaptive computation allocation, we
propose in-width and in-depth pruning strategies, featuring a light-weight
solution to achieve inference efficiency. Extensive experiments across seven
benchmarks show phi-Decoding outperforms strong baselines in both
performance and efficiency. Additional analysis demonstrates its generalization
across various LLMs and scalability across a wide range of computing budgets.
The code will be released at https://github.com/xufangzhi/phi-Decoding, and the
open-source PyPI package is coming soon.
A new decoding strategy, $\phi$-Decoding, uses foresight sampling and pruning techniques to enhance the efficiency and performance of large language models during inference.
foresight sampling $\phi$-Decoding decoding strategy step estimation simulated future steps optimal step adaptive computation allocation in-width pruning in-depth pruning LLMs scalability computing budgets