Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV
  Cache and Parallel Decoding
Diffusion-based large language models (Diffusion LLMs) have shown promise for
non-autoregressive text generation with parallel decoding capabilities.
However, the practical inference speed of open-sourced Diffusion LLMs often
lags behind autoregressive models due to the lack of Key-Value (KV) Cache and
quality degradation when decoding multiple tokens simultaneously. To bridge
this gap, we introduce a novel block-wise approximate KV Cache mechanism
tailored for bidirectional diffusion models, enabling cache reuse with
negligible performance drop. Additionally, we identify the root cause of
generation quality degradation in parallel decoding as the disruption of token
dependencies under the conditional independence assumption. To address this, we
propose a confidence-aware parallel decoding strategy that selectively decodes
tokens exceeding a confidence threshold, mitigating dependency violations and
maintaining generation quality. Experimental results on LLaDA and Dream models
across multiple LLM benchmarks demonstrate up to 27.6times
throughput improvement with minimal accuracy loss, closing the performance gap
with autoregressive models and paving the way for practical deployment of
Diffusion LLMs.
683931e1b6280677f75edf09
Chengyue Wu
683931e1b6280677f75edf0a
Hao Zhang
claimed_verified
2025-06-02T07:46:54.555Z
67ff3433a0346c2e622d70bf
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/HYKLHGWkUosIotYejiiS6.png
Hao Zhang
user
haozhcx
683931e1b6280677f75edf0b
Shuchen Xue
claimed_verified
2025-05-30T06:53:19.453Z
64cf5e81a2e7f9ff61eb3a0c
/avatars/cbaa11daec9d4113bf7de93fe9b9ee86.svg
scxue
user
Cauthyyy
683931e1b6280677f75edf0c
Zhijian Liu
claimed_verified
2025-05-30T06:53:22.124Z
650dac79b959b0e1d41d7378
https://cdn-avatars.huggingface.co/v1/production/uploads/650dac79b959b0e1d41d7378/mzbN0MFk3k8b94FQ40I7L.jpeg
Zhijian Liu
user
zhijianliu
683931e1b6280677f75edf0d
Shizhe Diao
683931e1b6280677f75edf0e
Ligeng Zhu
683931e1b6280677f75edf0f
Ping Luo
683931e1b6280677f75edf10
Song Han
683931e1b6280677f75edf11
Enze Xie
Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV
  Cache and Parallel Decoding
Diffusion-based large language models (Diffusion LLMs) have shown promise for
non-autoregressive text generation with parallel decoding capabilities.
However, the practical inference speed of open-sourced Diffusion LLMs often
lags behind autoregressive models due to the lack of Key-Value (KV) Cache and
quality degradation when decoding multiple tokens simultaneously. To bridge
this gap, we introduce a novel block-wise approximate KV Cache mechanism
tailored for bidirectional diffusion models, enabling cache reuse with
negligible performance drop. Additionally, we identify the root cause of
generation quality degradation in parallel decoding as the disruption of token
dependencies under the conditional independence assumption. To address this, we
propose a confidence-aware parallel decoding strategy that selectively decodes
tokens exceeding a confidence threshold, mitigating dependency violations and
maintaining generation quality. Experimental results on LLaDA and Dream models
across multiple LLM benchmarks demonstrate up to 27.6times
throughput improvement with minimal accuracy loss, closing the performance gap
with autoregressive models and paving the way for practical deployment of
Diffusion LLMs.
A novel block-wise approximate KV Cache and confidence-aware parallel decoding strategy improve the inference speed of diffusion-based large language models without significant quality loss.
diffusion-based large language models (Diffusion LLMs) non-autoregressive text generation parallel decoding Key-Value (KV) Cache bidirectional diffusion models token dependencies confidence-aware parallel decoding LLaDA Dream models LLM benchmarks