DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture
  Design in Text to Image Generation
In this work, we empirically study Diffusion Transformers (DiTs) for
text-to-image generation, focusing on architectural choices, text-conditioning
strategies, and training protocols. We evaluate a range of DiT-based
architectures--including PixArt-style and MMDiT variants--and compare them with
a standard DiT variant which directly processes concatenated text and noise
inputs. Surprisingly, our findings reveal that the performance of standard DiT
is comparable with those specialized models, while demonstrating superior
parameter-efficiency, especially when scaled up. Leveraging the layer-wise
parameter sharing strategy, we achieve a further reduction of 66% in model size
compared to an MMDiT architecture, with minimal performance impact. Building on
an in-depth analysis of critical components such as text encoders and
Variational Auto-Encoders (VAEs), we introduce DiT-Air and DiT-Air-Lite. With
supervised and reward fine-tuning, DiT-Air achieves state-of-the-art
performance on GenEval and T2I CompBench, while DiT-Air-Lite remains highly
competitive, surpassing most existing models despite its compact size.
67d3d2dec4a225b653154b3a
Chen Chen
claimed_verified
2025-03-16T21:14:18.789Z
65409e53d07194a81ecdd427
/avatars/73a88ffe7f78f05b6fc8eb57e9f1e587.svg
Chen Chen
user
ultra7chen
67d3d2dec4a225b653154b3b
Rui Qian
67d3d2dec4a225b653154b3c
Wenze Hu
admin_assigned
2025-03-14T10:51:01.388Z
638807bd5c2df0349f443de9
/avatars/8593af4c3b716dfe17227618873dd3f2.svg
wenze
user
wenzehu
67d3d2dec4a225b653154b3d
Tsu-Jui Fu
admin_assigned
2025-03-14T10:51:09.453Z
638cb2efc8912be69c149a46
https://cdn-avatars.huggingface.co/v1/production/uploads/638cb2efc8912be69c149a46/qxMNOWoSUG525DgNPpHcK.jpeg
Tsu-Jui Fu
user
tsujuifu
67d3d2dec4a225b653154b3e
Lezhi Li
67d3d2dec4a225b653154b3f
Bowen Zhang
67d3d2dec4a225b653154b40
Alex Schwing
67d3d2dec4a225b653154b41
Wei Liu
67d3d2dec4a225b653154b42
Yinfei Yang
admin_assigned
2025-03-14T10:50:24.690Z
64b762568c632fbca942a405
/avatars/1eb737ec169967872f1ebf5ff29f1e6b.svg
Yinfei Yang
user
yinfeiy
DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture
  Design in Text to Image Generation
In this work, we empirically study Diffusion Transformers (DiTs) for
text-to-image generation, focusing on architectural choices, text-conditioning
strategies, and training protocols. We evaluate a range of DiT-based
architectures--including PixArt-style and MMDiT variants--and compare them with
a standard DiT variant which directly processes concatenated text and noise
inputs. Surprisingly, our findings reveal that the performance of standard DiT
is comparable with those specialized models, while demonstrating superior
parameter-efficiency, especially when scaled up. Leveraging the layer-wise
parameter sharing strategy, we achieve a further reduction of 66% in model size
compared to an MMDiT architecture, with minimal performance impact. Building on
an in-depth analysis of critical components such as text encoders and
Variational Auto-Encoders (VAEs), we introduce DiT-Air and DiT-Air-Lite. With
supervised and reward fine-tuning, DiT-Air achieves state-of-the-art
performance on GenEval and T2I CompBench, while DiT-Air-Lite remains highly
competitive, surpassing most existing models despite its compact size.
Diffusion Transformers (DiTs) achieve competitive or superior text-to-image generation performance over specialized models while maintaining parameter efficiency.
Diffusion Transformers DiTs PixArt-style MMDiT text-conditioning training protocols parameter-efficiency layer-wise parameter sharing text encoders Variational Auto-Encoders VAEs DiT-Air DiT-Air-Lite GenEval T2I CompBench supervised fine-tuning reward fine-tuning