Learning Adaptive Parallel Reasoning with Language Models
Scaling inference-time computation has substantially improved the reasoning
capabilities of language models. However, existing methods have significant
limitations: serialized chain-of-thought approaches generate overly long
outputs, leading to increased latency and exhausted context windows, while
parallel methods such as self-consistency suffer from insufficient
coordination, resulting in redundant computations and limited performance
gains. To address these shortcomings, we propose Adaptive Parallel Reasoning
(APR), a novel reasoning framework that enables language models to orchestrate
both serialized and parallel computations end-to-end. APR generalizes existing
reasoning methods by enabling adaptive multi-threaded inference using spawn()
and join() operations. A key innovation is our end-to-end reinforcement
learning strategy, optimizing both parent and child inference threads to
enhance task success rate without requiring predefined reasoning structures.
Experiments on the Countdown reasoning task demonstrate significant benefits of
APR: (1) higher performance within the same context window (83.4% vs. 60.0% at
4k context); (2) superior scalability with increased computation (80.1% vs.
66.6% at 20k total tokens); (3) improved accuracy at equivalent latency (75.2%
vs. 57.3% at approximately 5,000ms). APR represents a step towards enabling
language models to autonomously optimize their reasoning processes through
adaptive allocation of computation.
6808480c49c8f78b6a4e492f
Jiayi Pan
admin_assigned
2025-04-23T13:46:37.415Z
61568f37272f2d87a99ba884
https://cdn-avatars.huggingface.co/v1/production/uploads/61568f37272f2d87a99ba884/lgvkl5f0rEyiQRVU5FE32.png
Jiayi Pan
user
Jiayi-Pan
6808480c49c8f78b6a4e4930
Xiuyu Li
claimed_verified
2025-04-23T08:27:59.248Z
644570ba2d91b15b4c7f6311
/avatars/d5e66012066d0c330b8f23718b1499d8.svg
Xiuyu Li
user
xiuyul
6808480c49c8f78b6a4e4931
Long Lian
admin_assigned
2025-04-23T13:46:44.955Z
63797c273f575acc2f6893c0
/avatars/32d7a6a8881c8c4d80a097b732ed24b6.svg
Long(Tony) Lian
user
longlian
6808480c49c8f78b6a4e4932
Charlie Snell
admin_assigned
2025-04-23T13:46:52.865Z
605d07fa8a3450814bada877
/avatars/eafe986982057fbaba962b99d5543477.svg
Charlie Snell
user
sea-snell
6808480c49c8f78b6a4e4933
Yifei Zhou
admin_assigned
2025-04-23T13:47:00.482Z
64fd0229e0dc35986bd3c0e5
/avatars/94f5698f9104dad7288edb4460026fd8.svg
Yifei Zhou
user
yifeizhou
6808480c49c8f78b6a4e4934
Adam Yala
claimed_verified
2025-04-23T08:28:02.029Z
6333a9195a032dcd095dda13
https://cdn-avatars.huggingface.co/v1/production/uploads/1664329996201-noauth.jpeg
Adam Yala
user
yala
6808480c49c8f78b6a4e4935
Trevor Darrell
admin_assigned
2025-04-23T13:47:06.847Z
64cbdf02f103036e23d1c7f3
/avatars/496069463900dea20929b57381182d39.svg
Trevor Darrell
user
trevordarrell
6808480c49c8f78b6a4e4936
Kurt Keutzer
admin_assigned
2025-04-23T13:47:13.477Z
6251bf4b183aa4266924ad91
https://cdn-avatars.huggingface.co/v1/production/uploads/1678041834400-6251bf4b183aa4266924ad91.jpeg
Kurt Keutzer
user
kurtkeutzer
6808480c49c8f78b6a4e4937
Alane Suhr
admin_assigned
2025-04-23T13:47:19.192Z
6611e6e1188ff298b0dd0b79
/avatars/3a495283955ec9e06e1829c7eb2cd9a4.svg
Alane Suhr
user
alsuhr
Learning Adaptive Parallel Reasoning with Language Models
Scaling inference-time computation has substantially improved the reasoning
capabilities of language models. However, existing methods have significant
limitations: serialized chain-of-thought approaches generate overly long
outputs, leading to increased latency and exhausted context windows, while
parallel methods such as self-consistency suffer from insufficient
coordination, resulting in redundant computations and limited performance
gains. To address these shortcomings, we propose Adaptive Parallel Reasoning
(APR), a novel reasoning framework that enables language models to orchestrate
both serialized and parallel computations end-to-end. APR generalizes existing
reasoning methods by enabling adaptive multi-threaded inference using spawn()
and join() operations. A key innovation is our end-to-end reinforcement
learning strategy, optimizing both parent and child inference threads to
enhance task success rate without requiring predefined reasoning structures.
Experiments on the Countdown reasoning task demonstrate significant benefits of
APR: (1) higher performance within the same context window (83.4% vs. 60.0% at
4k context); (2) superior scalability with increased computation (80.1% vs.
66.6% at 20k total tokens); (3) improved accuracy at equivalent latency (75.2%
vs. 57.3% at approximately 5,000ms). APR represents a step towards enabling
language models to autonomously optimize their reasoning processes through
adaptive allocation of computation.
Adaptive Parallel Reasoning (APR) enhances language model performance by optimally combining serialized and parallel computations through adaptive multi-threading and reinforcement learning.
Adaptive Parallel Reasoning APR serialized parallel computations spawn() join() reinforcement learning Countdown reasoning task multi-threaded inference context window scalability computation allocation