DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion
  Modeling
Diffusion Transformer (DiT), a promising diffusion model for visual
generation, demonstrates impressive performance but incurs significant
computational overhead. Intriguingly, analysis of pre-trained DiT models
reveals that global self-attention is often redundant, predominantly capturing
local patterns-highlighting the potential for more efficient alternatives. In
this paper, we revisit convolution as an alternative building block for
constructing efficient and expressive diffusion models. However, naively
replacing self-attention with convolution typically results in degraded
performance. Our investigations attribute this performance gap to the higher
channel redundancy in ConvNets compared to Transformers. To resolve this, we
introduce a compact channel attention mechanism that promotes the activation of
more diverse channels, thereby enhancing feature diversity. This leads to
Diffusion ConvNet (DiCo), a family of diffusion models built entirely from
standard ConvNet modules, offering strong generative performance with
significant efficiency gains. On class-conditional ImageNet benchmarks, DiCo
outperforms previous diffusion models in both image quality and generation
speed. Notably, DiCo-XL achieves an FID of 2.05 at 256x256 resolution and 2.53
at 512x512, with a 2.7x and 3.1x speedup over DiT-XL/2, respectively.
Furthermore, our largest model, DiCo-H, scaled to 1B parameters, reaches an FID
of 1.90 on ImageNet 256x256-without any additional supervision during training.
Code: https://github.com/shallowdream204/DiCo.
682a82f600248a93f52aa65f
Yuang Ai
682a82f600248a93f52aa660
Qihang Fan
682a82f600248a93f52aa661
Xuefeng Hu
claimed_verified
2025-05-26T08:23:03.134Z
65cb212725a3b8e44dc0a09a
/avatars/66c8457a8a8a85e676654479f6860a4b.svg
Xuefeng Hu
user
bytehxf
682a82f600248a93f52aa662
Zhenheng Yang
682a82f600248a93f52aa663
Ran He
682a82f600248a93f52aa664
Huaibo Huang
DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion
  Modeling
Diffusion Transformer (DiT), a promising diffusion model for visual
generation, demonstrates impressive performance but incurs significant
computational overhead. Intriguingly, analysis of pre-trained DiT models
reveals that global self-attention is often redundant, predominantly capturing
local patterns-highlighting the potential for more efficient alternatives. In
this paper, we revisit convolution as an alternative building block for
constructing efficient and expressive diffusion models. However, naively
replacing self-attention with convolution typically results in degraded
performance. Our investigations attribute this performance gap to the higher
channel redundancy in ConvNets compared to Transformers. To resolve this, we
introduce a compact channel attention mechanism that promotes the activation of
more diverse channels, thereby enhancing feature diversity. This leads to
Diffusion ConvNet (DiCo), a family of diffusion models built entirely from
standard ConvNet modules, offering strong generative performance with
significant efficiency gains. On class-conditional ImageNet benchmarks, DiCo
outperforms previous diffusion models in both image quality and generation
speed. Notably, DiCo-XL achieves an FID of 2.05 at 256x256 resolution and 2.53
at 512x512, with a 2.7x and 3.1x speedup over DiT-XL/2, respectively.
Furthermore, our largest model, DiCo-H, scaled to 1B parameters, reaches an FID
of 1.90 on ImageNet 256x256-without any additional supervision during training.
Code: https://github.com/shallowdream204/DiCo.
Diffusion ConvNet (DiCo), using standard ConvNet modules with a compact channel attention mechanism, achieves high image quality and generation speed in visual generation tasks with efficiency gains compared to Diffusion Transformer (DiT).
diffusion model visual generation global self-attention convolution performance gap channel redundancy channel attention mechanism Diffusion ConvNet class-conditional ImageNet benchmarks FID speedup