title: SNOOPI: Supercharged One-step Diffusion Distillation with Proper
    Guidance
summary: Recent approaches have yielded promising results in distilling multi-step
  text-to-image diffusion models into one-step ones. The state-of-the-art
  efficient distillation technique, i.e., SwiftBrushv2 (SBv2), even surpasses the
  teacher model's performance with limited resources. However, our study reveals
  its instability when handling different diffusion model backbones due to using
  a fixed guidance scale within the Variational Score Distillation (VSD) loss.
  Another weakness of the existing one-step diffusion models is the missing
  support for negative prompt guidance, which is crucial in practical image
  generation. This paper presents SNOOPI, a novel framework designed to address
  these limitations by enhancing the guidance in one-step diffusion models during
  both training and inference. First, we effectively enhance training stability
  through Proper Guidance-SwiftBrush (PG-SB), which employs a random-scale
  classifier-free guidance approach. By varying the guidance scale of both
  teacher models, we broaden their output distributions, resulting in a more
  robust VSD loss that enables SB to perform effectively across diverse backbones
  while maintaining competitive performance. Second, we propose a training-free
  method called Negative-Away Steer Attention (NASA), which integrates negative
  prompts into one-step diffusion models via cross-attention to suppress
  undesired elements in generated images. Our experimental results show that our
  proposed methods significantly improve baseline models across various metrics.
  Remarkably, we achieve an HPSv2 score of 31.08, setting a new state-of-the-art
  benchmark for one-step diffusion models.
paper_authors._id: 674fd4fe696ab96e57021fbf
paper_authors.name: Viet Nguyen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-04T09:44:28.182Z
paper_authors._id: 643684cdc81d1646aa6b93a5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643684cdc81d1646aa6b93a5/iyLQfKMBCJJjfOJB2zvfp.png
paper_authors.fullname: Viet Nguyen
paper_authors.type: user
paper_authors.user: viettmab
paper_authors._id: 674fd4fe696ab96e57021fc0
paper_authors.name: Anh Aengus Nguyen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-04T09:44:30.839Z
paper_authors._id: 6397329eb3d9042831c885af
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6397329eb3d9042831c885af/sEIXpwBDv6dl5iUdsjA1I.jpeg
paper_authors.fullname: Anh Nguyen (Aengus)
paper_authors.type: user
paper_authors.user: aengusng
paper_authors._id: 674fd4fe696ab96e57021fc1
paper_authors.name: Trung Dao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-05T09:18:50.613Z
paper_authors._id: 64d265019badd06a0587c848
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64d265019badd06a0587c848/qrOtJ7gxUCzgVwduFDktf.jpeg
paper_authors.fullname: Dao Tuan Trung
paper_authors.type: user
paper_authors.user: termanteus
paper_authors._id: 674fd4fe696ab96e57021fc2
paper_authors.name: Khoi Nguyen
paper_authors._id: 674fd4fe696ab96e57021fc3
paper_authors.name: Cuong Pham
paper_authors._id: 674fd4fe696ab96e57021fc4
paper_authors.name: Toan Tran
paper_authors._id: 674fd4fe696ab96e57021fc5
paper_authors.name: Anh Tran
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-05T09:18:48.756Z
paper_authors._id: 63e9fc6174f940d171e4c5e9
paper_authors.avatarUrl: /avatars/0c34fdaecd7bd8aaffb3d68e83c0eeaf.svg
paper_authors.fullname: Anh Tran
paper_authors.type: user
paper_authors.user: anhttran1111
paper_title: SNOOPI: Supercharged One-step Diffusion Distillation with Proper
    Guidance
paper_summary: Recent approaches have yielded promising results in distilling multi-step
  text-to-image diffusion models into one-step ones. The state-of-the-art
  efficient distillation technique, i.e., SwiftBrushv2 (SBv2), even surpasses the
  teacher model's performance with limited resources. However, our study reveals
  its instability when handling different diffusion model backbones due to using
  a fixed guidance scale within the Variational Score Distillation (VSD) loss.
  Another weakness of the existing one-step diffusion models is the missing
  support for negative prompt guidance, which is crucial in practical image
  generation. This paper presents SNOOPI, a novel framework designed to address
  these limitations by enhancing the guidance in one-step diffusion models during
  both training and inference. First, we effectively enhance training stability
  through Proper Guidance-SwiftBrush (PG-SB), which employs a random-scale
  classifier-free guidance approach. By varying the guidance scale of both
  teacher models, we broaden their output distributions, resulting in a more
  robust VSD loss that enables SB to perform effectively across diverse backbones
  while maintaining competitive performance. Second, we propose a training-free
  method called Negative-Away Steer Attention (NASA), which integrates negative
  prompts into one-step diffusion models via cross-attention to suppress
  undesired elements in generated images. Our experimental results show that our
  proposed methods significantly improve baseline models across various metrics.
  Remarkably, we achieve an HPSv2 score of 31.08, setting a new state-of-the-art
  benchmark for one-step diffusion models.
paper_ai_summary: SNOOPI framework improves one-step text-to-image diffusion models with Proper Guidance-SwiftBrush and Negative-Away Steer Attention, enhancing both training stability and negative prompt support.
paper_ai_keywords: text-to-image diffusion models SwiftBrushv2 Variational Score Distillation Proper Guidance-SwiftBrush Negative-Away Steer Attention cross-attention HPSv2