title: Repulsive Score Distillation for Diverse Sampling of Diffusion Models
summary: Score distillation sampling has been pivotal for integrating diffusion models
  into generation of complex visuals. Despite impressive results it suffers from
  mode collapse and lack of diversity. To cope with this challenge, we leverage
  the gradient flow interpretation of score distillation to propose Repulsive
  Score Distillation (RSD). In particular, we propose a variational framework
  based on repulsion of an ensemble of particles that promotes diversity. Using a
  variational approximation that incorporates a coupling among particles, the
  repulsion appears as a simple regularization that allows interaction of
  particles based on their relative pairwise similarity, measured e.g., via
  radial basis kernels. We design RSD for both unconstrained and constrained
  sampling scenarios. For constrained sampling we focus on inverse problems in
  the latent space that leads to an augmented variational formulation, that
  strikes a good balance between compute, quality and diversity. Our extensive
  experiments for text-to-image generation, and inverse problems demonstrate that
  RSD achieves a superior trade-off between diversity and quality compared with
  state-of-the-art alternatives.
paper_authors._id: 667b3187d24677e9af84110a
paper_authors.name: Nicolas Zilberstein
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-25T22:53:12.740Z
paper_authors._id: 6635bc694b8c6ce85cc10c75
paper_authors.avatarUrl: /avatars/ddbdfa1940d2858065d62b7ac597b48a.svg
paper_authors.fullname: Zilberstein
paper_authors.type: user
paper_authors.user: nicozilber
paper_authors._id: 667b3187d24677e9af84110b
paper_authors.name: Morteza Mardani
paper_authors._id: 667b3187d24677e9af84110c
paper_authors.name: Santiago Segarra
paper_title: Repulsive Score Distillation for Diverse Sampling of Diffusion Models
paper_summary: Score distillation sampling has been pivotal for integrating diffusion models
  into generation of complex visuals. Despite impressive results it suffers from
  mode collapse and lack of diversity. To cope with this challenge, we leverage
  the gradient flow interpretation of score distillation to propose Repulsive
  Score Distillation (RSD). In particular, we propose a variational framework
  based on repulsion of an ensemble of particles that promotes diversity. Using a
  variational approximation that incorporates a coupling among particles, the
  repulsion appears as a simple regularization that allows interaction of
  particles based on their relative pairwise similarity, measured e.g., via
  radial basis kernels. We design RSD for both unconstrained and constrained
  sampling scenarios. For constrained sampling we focus on inverse problems in
  the latent space that leads to an augmented variational formulation, that
  strikes a good balance between compute, quality and diversity. Our extensive
  experiments for text-to-image generation, and inverse problems demonstrate that
  RSD achieves a superior trade-off between diversity and quality compared with
  state-of-the-art alternatives.
paper_ai_keywords: score distillation sampling diffusion models generation of complex visuals mode collapse lack of diversity gradient flow interpretation Repulsive Score Distillation (RSD) variational framework ensemble of particles variational approximation coupling among particles repulsion radial basis kernels unconstrained sampling constrained sampling inverse problems latent space augmented variational Formulation Text-to-image generation diversity quality