title: Bigger is not Always Better: Scaling Properties of Latent Diffusion
    Models
summary: We study the scaling properties of latent diffusion models (LDMs) with an
  emphasis on their sampling efficiency. While improved network architecture and
  inference algorithms have shown to effectively boost sampling efficiency of
  diffusion models, the role of model size -- a critical determinant of sampling
  efficiency -- has not been thoroughly examined. Through empirical analysis of
  established text-to-image diffusion models, we conduct an in-depth
  investigation into how model size influences sampling efficiency across varying
  sampling steps. Our findings unveil a surprising trend: when operating under a
  given inference budget, smaller models frequently outperform their larger
  equivalents in generating high-quality results. Moreover, we extend our study
  to demonstrate the generalizability of the these findings by applying various
  diffusion samplers, exploring diverse downstream tasks, evaluating
  post-distilled models, as well as comparing performance relative to training
  compute. These findings open up new pathways for the development of LDM scaling
  strategies which can be employed to enhance generative capabilities within
  limited inference budgets.
paper_authors._id: 660cca736601973e7a2fdc98
paper_authors.name: Kangfu Mei
paper_authors._id: 660cca736601973e7a2fdc99
paper_authors.name: Zhengzhong Tu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T09:18:06.980Z
paper_authors._id: 62548d5fef3debb2ddf91217
paper_authors.avatarUrl: /avatars/14975b45568f9c399c92c3986b6ce83e.svg
paper_authors.fullname: Zhengzhong Tu
paper_authors.type: user
paper_authors.user: vztu
paper_authors._id: 660cca736601973e7a2fdc9a
paper_authors.name: Mauricio Delbracio
paper_authors._id: 660cca736601973e7a2fdc9b
paper_authors.name: Hossein Talebi
paper_authors._id: 660cca736601973e7a2fdc9c
paper_authors.name: Vishal M. Patel
paper_authors._id: 660cca736601973e7a2fdc9d
paper_authors.name: Peyman Milanfar
paper_title: Bigger is not Always Better: Scaling Properties of Latent Diffusion
    Models
paper_summary: We study the scaling properties of latent diffusion models (LDMs) with an
  emphasis on their sampling efficiency. While improved network architecture and
  inference algorithms have shown to effectively boost sampling efficiency of
  diffusion models, the role of model size -- a critical determinant of sampling
  efficiency -- has not been thoroughly examined. Through empirical analysis of
  established text-to-image diffusion models, we conduct an in-depth
  investigation into how model size influences sampling efficiency across varying
  sampling steps. Our findings unveil a surprising trend: when operating under a
  given inference budget, smaller models frequently outperform their larger
  equivalents in generating high-quality results. Moreover, we extend our study
  to demonstrate the generalizability of the these findings by applying various
  diffusion samplers, exploring diverse downstream tasks, evaluating
  post-distilled models, as well as comparing performance relative to training
  compute. These findings open up new pathways for the development of LDM scaling
  strategies which can be employed to enhance generative capabilities within
  limited inference budgets.
paper_ai_summary: Smaller latent diffusion models often outperform larger ones in terms of sampling efficiency and generate high-quality results within limited inference budgets.
paper_ai_keywords: latent diffusion models sampling efficiency diffusion models model size inference budget sampling steps diffusion samplers post-distilled models generative capabilities