title: SmoothCache: A Universal Inference Acceleration Technique for Diffusion
    Transformers
summary: Diffusion Transformers (DiT) have emerged as powerful generative models for
  various tasks, including image, video, and speech synthesis. However, their
  inference process remains computationally expensive due to the repeated
  evaluation of resource-intensive attention and feed-forward modules. To address
  this, we introduce SmoothCache, a model-agnostic inference acceleration
  technique for DiT architectures. SmoothCache leverages the observed high
  similarity between layer outputs across adjacent diffusion timesteps. By
  analyzing layer-wise representation errors from a small calibration set,
  SmoothCache adaptively caches and reuses key features during inference. Our
  experiments demonstrate that SmoothCache achieves 8% to 71% speed up while
  maintaining or even improving generation quality across diverse modalities. We
  showcase its effectiveness on DiT-XL for image generation, Open-Sora for
  text-to-video, and Stable Audio Open for text-to-audio, highlighting its
  potential to enable real-time applications and broaden the accessibility of
  powerful DiT models.
paper_authors._id: 673c2472d1794741928fa132
paper_authors.name: Joseph Liu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-19T05:39:04.787Z
paper_authors._id: 65330e31d690f3012e0557a2
paper_authors.avatarUrl: /avatars/c0969c9643a6dc70ff7b3e65c9412e54.svg
paper_authors.fullname: Eng Ann Joseph Liu
paper_authors.type: user
paper_authors.user: josephliu-roblox
paper_authors._id: 673c2472d1794741928fa133
paper_authors.name: Joshua Geddes
paper_authors._id: 673c2472d1794741928fa134
paper_authors.name: Ziyu Guo
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-19T05:39:04.787Z
paper_authors._id: 648cce407f7821f063c140be
paper_authors.avatarUrl: /avatars/6a458c35f34bfbceba6be8e5b6fb1437.svg
paper_authors.fullname: Ziyu Guo
paper_authors.type: user
paper_authors.user: helloterran
paper_authors._id: 673c2472d1794741928fa135
paper_authors.name: Haomiao Jiang
paper_authors._id: 673c2472d1794741928fa136
paper_authors.name: Mahesh Kumar Nandwana
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-19T05:39:04.787Z
paper_authors._id: 667f437aa94430cd2e723346
paper_authors.avatarUrl: /avatars/c7d71f6789756082e129b0b9cf9dcfda.svg
paper_authors.fullname: Mahesh Nandwana
paper_authors.type: user
paper_authors.user: mnandwana
paper_title: SmoothCache: A Universal Inference Acceleration Technique for Diffusion
    Transformers
paper_summary: Diffusion Transformers (DiT) have emerged as powerful generative models for
  various tasks, including image, video, and speech synthesis. However, their
  inference process remains computationally expensive due to the repeated
  evaluation of resource-intensive attention and feed-forward modules. To address
  this, we introduce SmoothCache, a model-agnostic inference acceleration
  technique for DiT architectures. SmoothCache leverages the observed high
  similarity between layer outputs across adjacent diffusion timesteps. By
  analyzing layer-wise representation errors from a small calibration set,
  SmoothCache adaptively caches and reuses key features during inference. Our
  experiments demonstrate that SmoothCache achieves 8% to 71% speed up while
  maintaining or even improving generation quality across diverse modalities. We
  showcase its effectiveness on DiT-XL for image generation, Open-Sora for
  text-to-video, and Stable Audio Open for text-to-audio, highlighting its
  potential to enable real-time applications and broaden the accessibility of
  powerful DiT models.
paper_ai_summary: SmoothCache accelerates DiT inference by caching key features, achieving significant speed-ups while maintaining quality across image, video, and audio tasks.
paper_ai_keywords: Diffusion Transformers DiT SmoothCache inference acceleration attention modules feed-forward modules layer-wise representation errors real-time applications