Accelerating Diffusion LLMs via Adaptive Parallel Decoding
The generation speed of LLMs are bottlenecked by autoregressive decoding,
where tokens are predicted sequentially one by one. Alternatively, diffusion
large language models (dLLMs) theoretically allow for parallel token
generation, but in practice struggle to achieve the speed of autoregressive
models without significantly sacrificing quality. We therefore introduce
adaptive parallel decoding (APD), a novel method that dynamically adjusts the
number of tokens sampled in parallel. We achieve this by defining a
multiplicative mixture between the dLLM marginal probabilities and the joint
probability of sequences under a small auxiliary autoregressive model. This
inverts the standard setup of speculative decoding, where the goal is to sample
from a large autoregressive verifier by drafting from a smaller model. We
further optimize APD by enabling KV caching and limiting the size of the masked
input. Altogether, our method puts forward three tunable parameters to flexibly
tradeoff throughput and quality. We show that APD provides markedly higher
throughput with minimal quality degradations on downstream benchmarks.
683e703c33ac56778c2e51cd
Daniel Israel
claimed_verified
2025-06-03T08:45:22.988Z
630139f1f6bea7dd15bdaf4e
/avatars/263536f6160f8c522d2a76ca2c4e4cc0.svg
Daniel Israel
user
danielmisrael
683e703c33ac56778c2e51ce
Guy Van den Broeck
683e703c33ac56778c2e51cf
Aditya Grover
Accelerating Diffusion LLMs via Adaptive Parallel Decoding
The generation speed of LLMs are bottlenecked by autoregressive decoding,
where tokens are predicted sequentially one by one. Alternatively, diffusion
large language models (dLLMs) theoretically allow for parallel token
generation, but in practice struggle to achieve the speed of autoregressive
models without significantly sacrificing quality. We therefore introduce
adaptive parallel decoding (APD), a novel method that dynamically adjusts the
number of tokens sampled in parallel. We achieve this by defining a
multiplicative mixture between the dLLM marginal probabilities and the joint
probability of sequences under a small auxiliary autoregressive model. This
inverts the standard setup of speculative decoding, where the goal is to sample
from a large autoregressive verifier by drafting from a smaller model. We
further optimize APD by enabling KV caching and limiting the size of the masked
input. Altogether, our method puts forward three tunable parameters to flexibly
tradeoff throughput and quality. We show that APD provides markedly higher
throughput with minimal quality degradations on downstream benchmarks.
Adaptive parallel decoding (APD) enhances the throughput of diffusion large language models (dLLMs) by dynamically adjusting parallel token generation without significantly diminishing quality.
autoregressive decoding diffusion large language models dLLMs adaptive parallel decoding APD marginal probabilities joint probability speculative decoding KV caching masked input