title: Score Distillation Sampling with Learned Manifold Corrective
summary: Score Distillation Sampling (SDS) is a recent but already widely popular
  method that relies on an image diffusion model to control optimization problems
  using text prompts. In this paper, we conduct an in-depth analysis of the SDS
  loss function, identify an inherent problem with its formulation, and propose a
  surprisingly easy but effective fix. Specifically, we decompose the loss into
  different factors and isolate the component responsible for noisy gradients. In
  the original formulation, high text guidance is used to account for the noise,
  leading to unwanted side effects. Instead, we train a shallow network mimicking
  the timestep-dependent denoising deficiency of the image diffusion model in
  order to effectively factor it out. We demonstrate the versatility and the
  effectiveness of our novel loss formulation through several qualitative and
  quantitative experiments, including optimization-based image synthesis and
  editing, zero-shot image translation network training, and text-to-3D
  synthesis.
paper_authors._id: 659f6862874e583fed675471
paper_authors.name: Thiemo Alldieck
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T11:20:20.428Z
paper_authors._id: 648c964d39d2584ee47af19c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/648c964d39d2584ee47af19c/5UEkzDTMY8I3svKjR7kxN.jpeg
paper_authors.fullname: Thiemo Alldieck
paper_authors.type: user
paper_authors.user: thiemoall
paper_authors._id: 659f6862874e583fed675472
paper_authors.name: Nikos Kolotouros
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T11:20:28.010Z
paper_authors._id: 648cd29f8f46d7284544083b
paper_authors.avatarUrl: /avatars/53163398f1a629b9838548b808af53f7.svg
paper_authors.fullname: Nikos Kolotouros
paper_authors.type: user
paper_authors.user: kolotouros
paper_authors._id: 659f6862874e583fed675473
paper_authors.name: Cristian Sminchisescu
paper_title: Score Distillation Sampling with Learned Manifold Corrective
paper_summary: Score Distillation Sampling (SDS) is a recent but already widely popular
  method that relies on an image diffusion model to control optimization problems
  using text prompts. In this paper, we conduct an in-depth analysis of the SDS
  loss function, identify an inherent problem with its formulation, and propose a
  surprisingly easy but effective fix. Specifically, we decompose the loss into
  different factors and isolate the component responsible for noisy gradients. In
  the original formulation, high text guidance is used to account for the noise,
  leading to unwanted side effects. Instead, we train a shallow network mimicking
  the timestep-dependent denoising deficiency of the image diffusion model in
  order to effectively factor it out. We demonstrate the versatility and the
  effectiveness of our novel loss formulation through several qualitative and
  quantitative experiments, including optimization-based image synthesis and
  editing, zero-shot image translation network training, and text-to-3D
  synthesis.
paper_ai_summary: The paper addresses issues in the Score Distillation Sampling loss function by training a shallow network to remove noisy gradients introduced by high text guidance, demonstrating improvements in various applications like image synthesis, network training, and 3D synthesis.
paper_ai_keywords: image diffusion model Score Distillation Sampling (SDS) loss function noisy gradients timestep-dependent denoising deficiency optimization-based image synthesis zero-shot image translation text-to-3D synthesis