FineDiffusion: Scaling up Diffusion Models for Fine-grained Image Generation with 10,000 Classes
<|reference_start|>FineDiffusion: Scaling up Diffusion Models for Fine-grained Image Generation with 10,000 Classes: The class-conditional image generation based on diffusion models is renowned for generating high-quality and diverse images. However, most prior efforts focus on generating images for general categories, e.g., 1000 classes in ImageNet-1k. A more challenging task, large-scale fine-grained image generation, remains the boundary to explore. In this work, we present a parameter-efficient strategy, called FineDiffusion, to fine-tune large pre-trained diffusion models scaling to large-scale fine-grained image generation with 10,000 categories. FineDiffusion significantly accelerates training and reduces storage overhead by only fine-tuning tiered class embedder, bias terms, and normalization layers' parameters. To further improve the image generation quality of fine-grained categories, we propose a novel sampling method for fine-grained image generation, which utilizes superclass-conditioned guidance, specifically tailored for fine-grained categories, to replace the conventional classifier-free guidance sampling. Compared to full fine-tuning, FineDiffusion achieves a remarkable 1.56x training speed-up and requires storing merely 1.77% of the total model parameters, while achieving state-of-the-art FID of 9.776 on image generation of 10,000 classes. Extensive qualitative and quantitative experiments demonstrate the superiority of our method compared to other parameter-efficient fine-tuning methods. The code and more generated results are available at our project website: https://finediffusion.github.io/.<|reference_end|>
@article{pan2024finediffusion:,
    title={FineDiffusion: Scaling up Diffusion Models for Fine-grained Image
  Generation with 10,000 Classes},
    author={Ziying Pan, Kun Wang, Gang Li, Feihong He, Yongxuan Lai},
    journal={arXiv preprint arXiv:2402.18331},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.18331},
    primaryClass={cs.CV}
}