title: Eliminating Oversaturation and Artifacts of High Guidance Scales in
    Diffusion Models
summary: Classifier-free guidance (CFG) is crucial for improving both generation
  quality and alignment between the input condition and final output in diffusion
  models. While a high guidance scale is generally required to enhance these
  aspects, it also causes oversaturation and unrealistic artifacts. In this
  paper, we revisit the CFG update rule and introduce modifications to address
  this issue. We first decompose the update term in CFG into parallel and
  orthogonal components with respect to the conditional model prediction and
  observe that the parallel component primarily causes oversaturation, while the
  orthogonal component enhances image quality. Accordingly, we propose
  down-weighting the parallel component to achieve high-quality generations
  without oversaturation. Additionally, we draw a connection between CFG and
  gradient ascent and introduce a new rescaling and momentum method for the CFG
  update rule based on this insight. Our approach, termed adaptive projected
  guidance (APG), retains the quality-boosting advantages of CFG while enabling
  the use of higher guidance scales without oversaturation. APG is easy to
  implement and introduces practically no additional computational overhead to
  the sampling process. Through extensive experiments, we demonstrate that APG is
  compatible with various conditional diffusion models and samplers, leading to
  improved FID, recall, and saturation scores while maintaining precision
  comparable to CFG, making our method a superior plug-and-play alternative to
  standard classifier-free guidance.
paper_authors._id: 66ff8020001816b29eb1082d
paper_authors.name: Seyedmorteza Sadat
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T07:23:11.988Z
paper_authors._id: 63b4b02a103617b0a5b0ee2e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63b4b02a103617b0a5b0ee2e/LasBC-pCnPJ6XuCt6qqcp.jpeg
paper_authors.fullname: Seyedmorteza Sadat
paper_authors.type: user
paper_authors.user: msadat97
paper_authors._id: 66ff8020001816b29eb1082e
paper_authors.name: Otmar Hilliges
paper_authors._id: 66ff8020001816b29eb1082f
paper_authors.name: Romann M. Weber
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-04T08:35:04.385Z
paper_authors._id: 630f7646197cd3f24e7f8e9f
paper_authors.avatarUrl: /avatars/59bbd4ed38277b313051aac78f6808ac.svg
paper_authors.fullname: Romann Weber
paper_authors.type: user
paper_authors.user: RMW
paper_title: Eliminating Oversaturation and Artifacts of High Guidance Scales in
    Diffusion Models
paper_summary: Classifier-free guidance (CFG) is crucial for improving both generation
  quality and alignment between the input condition and final output in diffusion
  models. While a high guidance scale is generally required to enhance these
  aspects, it also causes oversaturation and unrealistic artifacts. In this
  paper, we revisit the CFG update rule and introduce modifications to address
  this issue. We first decompose the update term in CFG into parallel and
  orthogonal components with respect to the conditional model prediction and
  observe that the parallel component primarily causes oversaturation, while the
  orthogonal component enhances image quality. Accordingly, we propose
  down-weighting the parallel component to achieve high-quality generations
  without oversaturation. Additionally, we draw a connection between CFG and
  gradient ascent and introduce a new rescaling and momentum method for the CFG
  update rule based on this insight. Our approach, termed adaptive projected
  guidance (APG), retains the quality-boosting advantages of CFG while enabling
  the use of higher guidance scales without oversaturation. APG is easy to
  implement and introduces practically no additional computational overhead to
  the sampling process. Through extensive experiments, we demonstrate that APG is
  compatible with various conditional diffusion models and samplers, leading to
  improved FID, recall, and saturation scores while maintaining precision
  comparable to CFG, making our method a superior plug-and-play alternative to
  standard classifier-free guidance.
paper_ai_summary: Adaptive projected guidance modifies classifier-free guidance to enhance image quality and reduce oversaturation by decomposing and rescaling update components, enabling higher guidance scales without artifacts.
paper_ai_keywords: classifier-free guidance CFG diffusion models conditional model prediction parallel component orthogonal component oversaturation image quality gradient ascent adaptive projected guidance APG FID recall saturation scores precision