title: Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV
    Cache and Parallel Decoding
summary: Diffusion-based large language models (Diffusion LLMs) have shown promise for
  non-autoregressive text generation with parallel decoding capabilities.
  However, the practical inference speed of open-sourced Diffusion LLMs often
  lags behind autoregressive models due to the lack of Key-Value (KV) Cache and
  quality degradation when decoding multiple tokens simultaneously. To bridge
  this gap, we introduce a novel block-wise approximate KV Cache mechanism
  tailored for bidirectional diffusion models, enabling cache reuse with
  negligible performance drop. Additionally, we identify the root cause of
  generation quality degradation in parallel decoding as the disruption of token
  dependencies under the conditional independence assumption. To address this, we
  propose a confidence-aware parallel decoding strategy that selectively decodes
  tokens exceeding a confidence threshold, mitigating dependency violations and
  maintaining generation quality. Experimental results on LLaDA and Dream models
  across multiple LLM benchmarks demonstrate up to 27.6times
  throughput improvement with minimal accuracy loss, closing the performance gap
  with autoregressive models and paving the way for practical deployment of
  Diffusion LLMs.
paper_authors._id: 683931e1b6280677f75edf09
paper_authors.name: Chengyue Wu
paper_authors._id: 683931e1b6280677f75edf0a
paper_authors.name: Hao Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:46:54.555Z
paper_authors._id: 67ff3433a0346c2e622d70bf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/HYKLHGWkUosIotYejiiS6.png
paper_authors.fullname: Hao Zhang
paper_authors.type: user
paper_authors.user: haozhcx
paper_authors._id: 683931e1b6280677f75edf0b
paper_authors.name: Shuchen Xue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:53:19.453Z
paper_authors._id: 64cf5e81a2e7f9ff61eb3a0c
paper_authors.avatarUrl: /avatars/cbaa11daec9d4113bf7de93fe9b9ee86.svg
paper_authors.fullname: scxue
paper_authors.type: user
paper_authors.user: Cauthyyy
paper_authors._id: 683931e1b6280677f75edf0c
paper_authors.name: Zhijian Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:53:22.124Z
paper_authors._id: 650dac79b959b0e1d41d7378
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650dac79b959b0e1d41d7378/mzbN0MFk3k8b94FQ40I7L.jpeg
paper_authors.fullname: Zhijian Liu
paper_authors.type: user
paper_authors.user: zhijianliu
paper_authors._id: 683931e1b6280677f75edf0d
paper_authors.name: Shizhe Diao
paper_authors._id: 683931e1b6280677f75edf0e
paper_authors.name: Ligeng Zhu
paper_authors._id: 683931e1b6280677f75edf0f
paper_authors.name: Ping Luo
paper_authors._id: 683931e1b6280677f75edf10
paper_authors.name: Song Han
paper_authors._id: 683931e1b6280677f75edf11
paper_authors.name: Enze Xie
paper_title: Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV
    Cache and Parallel Decoding
paper_summary: Diffusion-based large language models (Diffusion LLMs) have shown promise for
  non-autoregressive text generation with parallel decoding capabilities.
  However, the practical inference speed of open-sourced Diffusion LLMs often
  lags behind autoregressive models due to the lack of Key-Value (KV) Cache and
  quality degradation when decoding multiple tokens simultaneously. To bridge
  this gap, we introduce a novel block-wise approximate KV Cache mechanism
  tailored for bidirectional diffusion models, enabling cache reuse with
  negligible performance drop. Additionally, we identify the root cause of
  generation quality degradation in parallel decoding as the disruption of token
  dependencies under the conditional independence assumption. To address this, we
  propose a confidence-aware parallel decoding strategy that selectively decodes
  tokens exceeding a confidence threshold, mitigating dependency violations and
  maintaining generation quality. Experimental results on LLaDA and Dream models
  across multiple LLM benchmarks demonstrate up to 27.6times
  throughput improvement with minimal accuracy loss, closing the performance gap
  with autoregressive models and paving the way for practical deployment of
  Diffusion LLMs.
paper_ai_summary: A novel block-wise approximate KV Cache and confidence-aware parallel decoding strategy improve the inference speed of diffusion-based large language models without significant quality loss.
paper_ai_keywords: diffusion-based large language models (Diffusion LLMs) non-autoregressive text generation parallel decoding Key-Value (KV) Cache bidirectional diffusion models token dependencies confidence-aware parallel decoding LLaDA Dream models LLM benchmarks