title: PLADIS: Pushing the Limits of Attention in Diffusion Models at Inference
    Time by Leveraging Sparsity
summary: Diffusion models have shown impressive results in generating high-quality
  conditional samples using guidance techniques such as Classifier-Free Guidance
  (CFG). However, existing methods often require additional training or neural
  function evaluations (NFEs), making them incompatible with guidance-distilled
  models. Also, they rely on heuristic approaches that need identifying target
  layers. In this work, we propose a novel and efficient method, termed PLADIS,
  which boosts pre-trained models (U-Net/Transformer) by leveraging sparse
  attention. Specifically, we extrapolate query-key correlations using softmax
  and its sparse counterpart in the cross-attention layer during inference,
  without requiring extra training or NFEs. By leveraging the noise robustness of
  sparse attention, our PLADIS unleashes the latent potential of text-to-image
  diffusion models, enabling them to excel in areas where they once struggled
  with newfound effectiveness. It integrates seamlessly with guidance techniques,
  including guidance-distilled models. Extensive experiments show notable
  improvements in text alignment and human preference, offering a highly
  efficient and universally applicable solution.
paper_authors._id: 67d2ca0767366130cccad93d
paper_authors.name: Kwanyoung Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:58:03.528Z
paper_authors._id: 63973ee44e7b4959dc98028f
paper_authors.avatarUrl: /avatars/2e166fee60844729479bfa4291796c8a.svg
paper_authors.fullname: Kwanyoung
paper_authors.type: user
paper_authors.user: kwanyoung
paper_authors._id: 67d2ca0767366130cccad93e
paper_authors.name: Byeongsu Sim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:47:11.835Z
paper_authors._id: 668377232d89090894bea7b4
paper_authors.avatarUrl: /avatars/1a74a08d645a352db4a460036b9fb6db.svg
paper_authors.fullname: byeongsu sim
paper_authors.type: user
paper_authors.user: byeongsus
paper_title: PLADIS: Pushing the Limits of Attention in Diffusion Models at Inference
    Time by Leveraging Sparsity
paper_summary: Diffusion models have shown impressive results in generating high-quality
  conditional samples using guidance techniques such as Classifier-Free Guidance
  (CFG). However, existing methods often require additional training or neural
  function evaluations (NFEs), making them incompatible with guidance-distilled
  models. Also, they rely on heuristic approaches that need identifying target
  layers. In this work, we propose a novel and efficient method, termed PLADIS,
  which boosts pre-trained models (U-Net/Transformer) by leveraging sparse
  attention. Specifically, we extrapolate query-key correlations using softmax
  and its sparse counterpart in the cross-attention layer during inference,
  without requiring extra training or NFEs. By leveraging the noise robustness of
  sparse attention, our PLADIS unleashes the latent potential of text-to-image
  diffusion models, enabling them to excel in areas where they once struggled
  with newfound effectiveness. It integrates seamlessly with guidance techniques,
  including guidance-distilled models. Extensive experiments show notable
  improvements in text alignment and human preference, offering a highly
  efficient and universally applicable solution.
paper_ai_summary: PLADIS leverages sparse attention in cross-attention layers to enhance pre-trained text-to-image diffusion models, improving text alignment and human preference without additional training.
paper_ai_keywords: diffusion models Classifier-Free Guidance (CFG) guidance-distilled models neural function evaluations (NFEs) PLADIS U-Net Transformer sparse attention softmax cross-attention layer text-to-image