title: D^2iT: Dynamic Diffusion Transformer for Accurate Image Generation
summary: Diffusion models are widely recognized for their ability to generate
  high-fidelity images. Despite the excellent performance and scalability of the
  Diffusion Transformer (DiT) architecture, it applies fixed compression across
  different image regions during the diffusion process, disregarding the
  naturally varying information densities present in these regions. However,
  large compression leads to limited local realism, while small compression
  increases computational complexity and compromises global consistency,
  ultimately impacting the quality of generated images. To address these
  limitations, we propose dynamically compressing different image regions by
  recognizing the importance of different regions, and introduce a novel
  two-stage framework designed to enhance the effectiveness and efficiency of
  image generation: (1) Dynamic VAE (DVAE) at first stage employs a hierarchical
  encoder to encode different image regions at different downsampling rates,
  tailored to their specific information densities, thereby providing more
  accurate and natural latent codes for the diffusion process. (2) Dynamic
  Diffusion Transformer (D^2iT) at second stage generates images by predicting
  multi-grained noise, consisting of coarse-grained (less latent code in smooth
  regions) and fine-grained (more latent codes in detailed regions), through an
  novel combination of the Dynamic Grain Transformer and the Dynamic Content
  Transformer. The strategy of combining rough prediction of noise with detailed
  regions correction achieves a unification of global consistency and local
  realism. Comprehensive experiments on various generation tasks validate the
  effectiveness of our approach. Code will be released at
  https://github.com/jiawn-creator/Dynamic-DiT.
paper_authors._id: 67ff6f8c661b74d0050d2774
paper_authors.name: Weinan Jia
paper_authors._id: 67ff6f8c661b74d0050d2775
paper_authors.name: Mengqi Huang
paper_authors._id: 67ff6f8c661b74d0050d2776
paper_authors.name: Nan Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T08:51:48.179Z
paper_authors._id: 6629d7c9fa14eaccf07d8633
paper_authors.avatarUrl: /avatars/dceb2f6c804c583adf15a3536c8c995b.svg
paper_authors.fullname: Nan Chen
paper_authors.type: user
paper_authors.user: CNcreator0331
paper_authors._id: 67ff6f8c661b74d0050d2777
paper_authors.name: Lei Zhang
paper_authors._id: 67ff6f8c661b74d0050d2778
paper_authors.name: Zhendong Mao
paper_title: D^2iT: Dynamic Diffusion Transformer for Accurate Image Generation
paper_summary: Diffusion models are widely recognized for their ability to generate
  high-fidelity images. Despite the excellent performance and scalability of the
  Diffusion Transformer (DiT) architecture, it applies fixed compression across
  different image regions during the diffusion process, disregarding the
  naturally varying information densities present in these regions. However,
  large compression leads to limited local realism, while small compression
  increases computational complexity and compromises global consistency,
  ultimately impacting the quality of generated images. To address these
  limitations, we propose dynamically compressing different image regions by
  recognizing the importance of different regions, and introduce a novel
  two-stage framework designed to enhance the effectiveness and efficiency of
  image generation: (1) Dynamic VAE (DVAE) at first stage employs a hierarchical
  encoder to encode different image regions at different downsampling rates,
  tailored to their specific information densities, thereby providing more
  accurate and natural latent codes for the diffusion process. (2) Dynamic
  Diffusion Transformer (D^2iT) at second stage generates images by predicting
  multi-grained noise, consisting of coarse-grained (less latent code in smooth
  regions) and fine-grained (more latent codes in detailed regions), through an
  novel combination of the Dynamic Grain Transformer and the Dynamic Content
  Transformer. The strategy of combining rough prediction of noise with detailed
  regions correction achieves a unification of global consistency and local
  realism. Comprehensive experiments on various generation tasks validate the
  effectiveness of our approach. Code will be released at
  https://github.com/jiawn-creator/Dynamic-DiT.
paper_ai_summary: A two-stage framework combining dynamic VAE encoding and dynamic diffusion transformer generation enhances image quality by adapting compression to regional information density.
paper_ai_keywords: diffusion models Diffusion Transformer Dynamic VAE hierarchical encoder downsampling rates information densities latent codes Dynamic Diffusion Transformer Dynamic Grain Transformer Dynamic Content Transformer multi-grained noise coarse-grained fine-grained global consistency local realism