title: Accelerating Diffusion LLMs via Adaptive Parallel Decoding
summary: The generation speed of LLMs are bottlenecked by autoregressive decoding,
  where tokens are predicted sequentially one by one. Alternatively, diffusion
  large language models (dLLMs) theoretically allow for parallel token
  generation, but in practice struggle to achieve the speed of autoregressive
  models without significantly sacrificing quality. We therefore introduce
  adaptive parallel decoding (APD), a novel method that dynamically adjusts the
  number of tokens sampled in parallel. We achieve this by defining a
  multiplicative mixture between the dLLM marginal probabilities and the joint
  probability of sequences under a small auxiliary autoregressive model. This
  inverts the standard setup of speculative decoding, where the goal is to sample
  from a large autoregressive verifier by drafting from a smaller model. We
  further optimize APD by enabling KV caching and limiting the size of the masked
  input. Altogether, our method puts forward three tunable parameters to flexibly
  tradeoff throughput and quality. We show that APD provides markedly higher
  throughput with minimal quality degradations on downstream benchmarks.
paper_authors._id: 683e703c33ac56778c2e51cd
paper_authors.name: Daniel Israel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:45:22.988Z
paper_authors._id: 630139f1f6bea7dd15bdaf4e
paper_authors.avatarUrl: /avatars/263536f6160f8c522d2a76ca2c4e4cc0.svg
paper_authors.fullname: Daniel Israel
paper_authors.type: user
paper_authors.user: danielmisrael
paper_authors._id: 683e703c33ac56778c2e51ce
paper_authors.name: Guy Van den Broeck
paper_authors._id: 683e703c33ac56778c2e51cf
paper_authors.name: Aditya Grover
paper_title: Accelerating Diffusion LLMs via Adaptive Parallel Decoding
paper_summary: The generation speed of LLMs are bottlenecked by autoregressive decoding,
  where tokens are predicted sequentially one by one. Alternatively, diffusion
  large language models (dLLMs) theoretically allow for parallel token
  generation, but in practice struggle to achieve the speed of autoregressive
  models without significantly sacrificing quality. We therefore introduce
  adaptive parallel decoding (APD), a novel method that dynamically adjusts the
  number of tokens sampled in parallel. We achieve this by defining a
  multiplicative mixture between the dLLM marginal probabilities and the joint
  probability of sequences under a small auxiliary autoregressive model. This
  inverts the standard setup of speculative decoding, where the goal is to sample
  from a large autoregressive verifier by drafting from a smaller model. We
  further optimize APD by enabling KV caching and limiting the size of the masked
  input. Altogether, our method puts forward three tunable parameters to flexibly
  tradeoff throughput and quality. We show that APD provides markedly higher
  throughput with minimal quality degradations on downstream benchmarks.
paper_ai_summary: Adaptive parallel decoding (APD) enhances the throughput of diffusion large language models (dLLMs) by dynamically adjusting parallel token generation without significantly diminishing quality.
paper_ai_keywords: autoregressive decoding diffusion large language models dLLMs adaptive parallel decoding APD marginal probabilities joint probability speculative decoding KV caching masked input