Scaling Speculative Decoding with Lookahead Reasoning
Reasoning models excel by generating long chain-of-thoughts, but decoding the
resulting thousands of tokens is slow. Token-level speculative decoding (SD)
helps, but its benefit is capped, because the chance that an entire
gamma-token guess is correct falls exponentially as gamma grows. This
means allocating more compute for longer token drafts faces an algorithmic
ceiling -- making the speedup modest and hardware-agnostic. We raise this
ceiling with Lookahead Reasoning, which exploits a second, step-level layer of
parallelism. Our key insight is that reasoning models generate step-by-step,
and each step needs only to be semantically correct, not exact token matching.
In Lookahead Reasoning, a lightweight draft model proposes several future
steps; the target model expands each proposal in one batched pass, and a
verifier keeps semantically correct steps while letting the target regenerate
any that fail. Token-level SD still operates within each reasoning step, so the
two layers of parallelism multiply. We show Lookahead Reasoning lifts the peak
speedup of SD both theoretically and empirically. Across GSM8K, AIME, and other
benchmarks, Lookahead Reasoning improves the speedup of SD from 1.4x to 2.1x
while preserving answer quality, and its speedup scales better with additional
GPU throughput. Our code is available at
https://github.com/hao-ai-lab/LookaheadReasoning
685c0361df8a0d6c70bbf91d
Yichao Fu
claimed_verified
2025-06-25T20:59:06.015Z
62ba66296501b0ff15ba1075
/avatars/de468727cb1240fd4b1f24a19fb237a6.svg
Yichao Fu
user
Viol2000
685c0361df8a0d6c70bbf91e
Rui Ge
685c0361df8a0d6c70bbf91f
Zelei Shao
685c0361df8a0d6c70bbf920
Zhijie Deng
685c0361df8a0d6c70bbf921
Hao Zhang
Scaling Speculative Decoding with Lookahead Reasoning
Reasoning models excel by generating long chain-of-thoughts, but decoding the
resulting thousands of tokens is slow. Token-level speculative decoding (SD)
helps, but its benefit is capped, because the chance that an entire
gamma-token guess is correct falls exponentially as gamma grows. This
means allocating more compute for longer token drafts faces an algorithmic
ceiling -- making the speedup modest and hardware-agnostic. We raise this
ceiling with Lookahead Reasoning, which exploits a second, step-level layer of
parallelism. Our key insight is that reasoning models generate step-by-step,
and each step needs only to be semantically correct, not exact token matching.
In Lookahead Reasoning, a lightweight draft model proposes several future
steps; the target model expands each proposal in one batched pass, and a
verifier keeps semantically correct steps while letting the target regenerate
any that fail. Token-level SD still operates within each reasoning step, so the
two layers of parallelism multiply. We show Lookahead Reasoning lifts the peak
speedup of SD both theoretically and empirically. Across GSM8K, AIME, and other
benchmarks, Lookahead Reasoning improves the speedup of SD from 1.4x to 2.1x
while preserving answer quality, and its speedup scales better with additional
GPU throughput. Our code is available at
https://github.com/hao-ai-lab/LookaheadReasoning
Lookahead Reasoning enhances the speed of speculative decoding by introducing step-level parallelism, improving speedup over token-level decoding while maintaining answer quality.
Lookahead Reasoning speculative decoding step-level parallelism token-level speculative decoding draft model target model verifier GSM8K AIME