title: The Superposition of Diffusion Models Using the Itô Density Estimator
summary: The Cambrian explosion of easily accessible pre-trained diffusion models
  suggests a demand for methods that combine multiple different pre-trained
  diffusion models without incurring the significant computational burden of
  re-training a larger combined model. In this paper, we cast the problem of
  combining multiple pre-trained diffusion models at the generation stage under a
  novel proposed framework termed superposition. Theoretically, we derive
  superposition from rigorous first principles stemming from the celebrated
  continuity equation and design two novel algorithms tailor-made for combining
  diffusion models in SuperDiff. SuperDiff leverages a new scalable It\^o density
  estimator for the log likelihood of the diffusion SDE which incurs no
  additional overhead compared to the well-known Hutchinson's estimator needed
  for divergence calculations. We demonstrate that SuperDiff is scalable to large
  pre-trained diffusion models as superposition is performed solely through
  composition during inference, and also enjoys painless implementation as it
  combines different pre-trained vector fields through an automated re-weighting
  scheme. Notably, we show that SuperDiff is efficient during inference time, and
  mimics traditional composition operators such as the logical OR and the logical
  AND. We empirically demonstrate the utility of using SuperDiff for generating
  more diverse images on CIFAR-10, more faithful prompt conditioned image editing
  using Stable Diffusion, and improved unconditional de novo structure design of
  proteins. https://github.com/necludov/super-diffusion
paper_authors._id: 676d0a2e0076ad5ba195b88a
paper_authors.name: Marta Skreta
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:32:04.997Z
paper_authors._id: 65a49ef27ec6af0f956a5c61
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65a49ef27ec6af0f956a5c61/OGJ1OVezooF7dUn1SiEKG.jpeg
paper_authors.fullname: Marta Skreta
paper_authors.type: user
paper_authors.user: mskrt
paper_authors._id: 676d0a2e0076ad5ba195b88b
paper_authors.name: Lazar Atanackovic
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:32:02.707Z
paper_authors._id: 66cd0e523f6dbc62985b184b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66cd0e523f6dbc62985b184b/vlHTY9LKJ0jKeAniHdGil.jpeg
paper_authors.fullname: Lazar Atanackovic
paper_authors.type: user
paper_authors.user: lazaratan
paper_authors._id: 676d0a2e0076ad5ba195b88c
paper_authors.name: Avishek Joey Bose
paper_authors._id: 676d0a2e0076ad5ba195b88d
paper_authors.name: Alexander Tong
paper_authors._id: 676d0a2e0076ad5ba195b88e
paper_authors.name: Kirill Neklyudov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T20:44:53.406Z
paper_authors._id: 65039d7a93574a897179139a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65039d7a93574a897179139a/szQAWkF2M9XnYKykmX8Cg.jpeg
paper_authors.fullname: Kirill Neklyudov
paper_authors.type: user
paper_authors.user: necludov
paper_title: The Superposition of Diffusion Models Using the Itô Density Estimator
paper_summary: The Cambrian explosion of easily accessible pre-trained diffusion models
  suggests a demand for methods that combine multiple different pre-trained
  diffusion models without incurring the significant computational burden of
  re-training a larger combined model. In this paper, we cast the problem of
  combining multiple pre-trained diffusion models at the generation stage under a
  novel proposed framework termed superposition. Theoretically, we derive
  superposition from rigorous first principles stemming from the celebrated
  continuity equation and design two novel algorithms tailor-made for combining
  diffusion models in SuperDiff. SuperDiff leverages a new scalable It\^o density
  estimator for the log likelihood of the diffusion SDE which incurs no
  additional overhead compared to the well-known Hutchinson's estimator needed
  for divergence calculations. We demonstrate that SuperDiff is scalable to large
  pre-trained diffusion models as superposition is performed solely through
  composition during inference, and also enjoys painless implementation as it
  combines different pre-trained vector fields through an automated re-weighting
  scheme. Notably, we show that SuperDiff is efficient during inference time, and
  mimics traditional composition operators such as the logical OR and the logical
  AND. We empirically demonstrate the utility of using SuperDiff for generating
  more diverse images on CIFAR-10, more faithful prompt conditioned image editing
  using Stable Diffusion, and improved unconditional de novo structure design of
  proteins. https://github.com/necludov/super-diffusion
paper_ai_keywords: diffusionDiffusion models Continuity equation Itô density estimator Hutchinson's estimator Composition Inference Superposition Re-weighting scheme Logical OR Logical AND SuperDiff FIDAR-10 Stable Diffusion Protein structure design