title: Region-Adaptive Sampling for Diffusion Transformers
summary: Diffusion models (DMs) have become the leading choice for generative tasks
  across diverse domains. However, their reliance on multiple sequential forward
  passes significantly limits real-time performance. Previous acceleration
  methods have primarily focused on reducing the number of sampling steps or
  reusing intermediate results, failing to leverage variations across spatial
  regions within the image due to the constraints of convolutional U-Net
  structures. By harnessing the flexibility of Diffusion Transformers (DiTs) in
  handling variable number of tokens, we introduce RAS, a novel, training-free
  sampling strategy that dynamically assigns different sampling ratios to regions
  within an image based on the focus of the DiT model. Our key observation is
  that during each sampling step, the model concentrates on semantically
  meaningful regions, and these areas of focus exhibit strong continuity across
  consecutive steps. Leveraging this insight, RAS updates only the regions
  currently in focus, while other regions are updated using cached noise from the
  previous step. The model's focus is determined based on the output from the
  preceding step, capitalizing on the temporal consistency we observed. We
  evaluate RAS on Stable Diffusion 3 and Lumina-Next-T2I, achieving speedups up
  to 2.36x and 2.51x, respectively, with minimal degradation in generation
  quality. Additionally, a user study reveals that RAS delivers comparable
  qualities under human evaluation while achieving a 1.6x speedup. Our approach
  makes a significant step towards more efficient diffusion transformers,
  enhancing their potential for real-time applications.
paper_authors._id: 67b2a89ebe31bfaa7cd2bff1
paper_authors.name: Ziming Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:33:14.074Z
paper_authors._id: 632924029c3f42ca7149f305
paper_authors.avatarUrl: /avatars/080bc7da4ad2875bdfa359213c88feb7.svg
paper_authors.fullname: Liu Ziming
paper_authors.type: user
paper_authors.user: MaruyamaAya
paper_authors._id: 67b2a89ebe31bfaa7cd2bff2
paper_authors.name: Yifan Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:33:16.182Z
paper_authors._id: 62d18eb81e36881a57f29bf4
paper_authors.avatarUrl: /avatars/104851421b4ee9641daaf15942fa7ea1.svg
paper_authors.fullname: Yif Yang
paper_authors.type: user
paper_authors.user: Yif29
paper_authors._id: 67b2a89ebe31bfaa7cd2bff3
paper_authors.name: Chengruidong Zhang
paper_authors._id: 67b2a89ebe31bfaa7cd2bff4
paper_authors.name: Yiqi Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:33:07.641Z
paper_authors._id: 6356bf52e983d3c51d212205
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1666629451430-noauth.jpeg
paper_authors.fullname: Yiqi Zhang
paper_authors.type: user
paper_authors.user: Viscent
paper_authors._id: 67b2a89ebe31bfaa7cd2bff5
paper_authors.name: Lili Qiu
paper_authors._id: 67b2a89ebe31bfaa7cd2bff6
paper_authors.name: Yang You
paper_authors._id: 67b2a89ebe31bfaa7cd2bff7
paper_authors.name: Yuqing Yang
paper_title: Region-Adaptive Sampling for Diffusion Transformers
paper_summary: Diffusion models (DMs) have become the leading choice for generative tasks
  across diverse domains. However, their reliance on multiple sequential forward
  passes significantly limits real-time performance. Previous acceleration
  methods have primarily focused on reducing the number of sampling steps or
  reusing intermediate results, failing to leverage variations across spatial
  regions within the image due to the constraints of convolutional U-Net
  structures. By harnessing the flexibility of Diffusion Transformers (DiTs) in
  handling variable number of tokens, we introduce RAS, a novel, training-free
  sampling strategy that dynamically assigns different sampling ratios to regions
  within an image based on the focus of the DiT model. Our key observation is
  that during each sampling step, the model concentrates on semantically
  meaningful regions, and these areas of focus exhibit strong continuity across
  consecutive steps. Leveraging this insight, RAS updates only the regions
  currently in focus, while other regions are updated using cached noise from the
  previous step. The model's focus is determined based on the output from the
  preceding step, capitalizing on the temporal consistency we observed. We
  evaluate RAS on Stable Diffusion 3 and Lumina-Next-T2I, achieving speedups up
  to 2.36x and 2.51x, respectively, with minimal degradation in generation
  quality. Additionally, a user study reveals that RAS delivers comparable
  qualities under human evaluation while achieving a 1.6x speedup. Our approach
  makes a significant step towards more efficient diffusion transformers,
  enhancing their potential for real-time applications.
paper_ai_keywords: Diffusion Transformers (DiTs) tokens sampling strategy sampling ratios regions focus sampling steps semantically meaningful temporal consistency cached noise Stable Diffusion Lumina-Next-T2I