Upsample What Matters: Region-Adaptive Latent Sampling for Accelerated
  Diffusion Transformers
Diffusion transformers have emerged as an alternative to U-net-based
diffusion models for high-fidelity image and video generation, offering
superior scalability. However, their heavy computation remains a major obstacle
to real-world deployment. Existing acceleration methods primarily exploit the
temporal dimension such as reusing cached features across diffusion timesteps.
Here, we propose Region-Adaptive Latent Upsampling (RALU), a training-free
framework that accelerates inference along spatial dimension. RALU performs
mixed-resolution sampling across three stages: 1) low-resolution denoising
latent diffusion to efficiently capture global semantic structure, 2)
region-adaptive upsampling on specific regions prone to artifacts at
full-resolution, and 3) all latent upsampling at full-resolution for detail
refinement. To stabilize generations across resolution transitions, we leverage
noise-timestep rescheduling to adapt the noise level across varying
resolutions. Our method significantly reduces computation while preserving
image quality by achieving up to 7.0times speed-up on FLUX and 3.0times
on Stable Diffusion 3 with minimal degradation. Furthermore, RALU is
complementary to existing temporal accelerations such as caching methods, thus
can be seamlessly integrated to further reduce inference latency without
compromising generation quality.
6880820200f4b5a05f2fbeb1
Wongi Jeong
claimed_verified
2025-07-23T08:36:07.672Z
63a1684281173de5e4caf63b
/avatars/68ac507ffa3ff7cb6899b8c8d002c01f.svg
wongi
user
ignow
6880820200f4b5a05f2fbeb2
Kyungryeol Lee
claimed_verified
2025-07-23T08:02:17.082Z
67076244679d36bc0e3eda5b
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0yE8cHzBuz9QA7sE8rQkf.png
Kyungryeol Lee
user
hirussell
6880820200f4b5a05f2fbeb3
Hoigi Seo
claimed_verified
2025-07-23T08:02:18.825Z
633e6f07309a99325095dd42
/avatars/57b91a488ac1745b3c0509c04eb6ad93.svg
Hoigi Seo
user
Agorium
6880820200f4b5a05f2fbeb4
Se Young Chun
Upsample What Matters: Region-Adaptive Latent Sampling for Accelerated
  Diffusion Transformers
Diffusion transformers have emerged as an alternative to U-net-based
diffusion models for high-fidelity image and video generation, offering
superior scalability. However, their heavy computation remains a major obstacle
to real-world deployment. Existing acceleration methods primarily exploit the
temporal dimension such as reusing cached features across diffusion timesteps.
Here, we propose Region-Adaptive Latent Upsampling (RALU), a training-free
framework that accelerates inference along spatial dimension. RALU performs
mixed-resolution sampling across three stages: 1) low-resolution denoising
latent diffusion to efficiently capture global semantic structure, 2)
region-adaptive upsampling on specific regions prone to artifacts at
full-resolution, and 3) all latent upsampling at full-resolution for detail
refinement. To stabilize generations across resolution transitions, we leverage
noise-timestep rescheduling to adapt the noise level across varying
resolutions. Our method significantly reduces computation while preserving
image quality by achieving up to 7.0times speed-up on FLUX and 3.0times
on Stable Diffusion 3 with minimal degradation. Furthermore, RALU is
complementary to existing temporal accelerations such as caching methods, thus
can be seamlessly integrated to further reduce inference latency without
compromising generation quality.
Region-Adaptive Latent Upsampling (RALU) accelerates diffusion transformer inference by performing mixed-resolution sampling, reducing computation while maintaining image quality.
diffusion transformers U-net-based diffusion models high-fidelity image and video generation scalability temporal dimension cached features diffusion timesteps Region-Adaptive Latent Upsampling mixed-resolution sampling low-resolution denoising latent diffusion region-adaptive upsampling full-resolution detail refinement noise-timestep rescheduling FLUX Stable Diffusion 3 inference latency