title: Be Yourself: Bounded Attention for Multi-Subject Text-to-Image
    Generation
summary: Text-to-image diffusion models have an unprecedented ability to generate
  diverse and high-quality images. However, they often struggle to faithfully
  capture the intended semantics of complex input prompts that include multiple
  subjects. Recently, numerous layout-to-image extensions have been introduced to
  improve user control, aiming to localize subjects represented by specific
  tokens. Yet, these methods often produce semantically inaccurate images,
  especially when dealing with multiple semantically or visually similar
  subjects. In this work, we study and analyze the causes of these limitations.
  Our exploration reveals that the primary issue stems from inadvertent semantic
  leakage between subjects in the denoising process. This leakage is attributed
  to the diffusion model's attention layers, which tend to blend the visual
  features of different subjects. To address these issues, we introduce Bounded
  Attention, a training-free method for bounding the information flow in the
  sampling process. Bounded Attention prevents detrimental leakage among subjects
  and enables guiding the generation to promote each subject's individuality,
  even with complex multi-subject conditioning. Through extensive
  experimentation, we demonstrate that our method empowers the generation of
  multiple subjects that better align with given prompts and layouts.
paper_authors._id: 66024830c618afa8d796e49c
paper_authors.name: Omer Dahary
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-26T09:21:35.365Z
paper_authors._id: 64ff145e1a7cb7c342ae0c53
paper_authors.avatarUrl: /avatars/5f1b8e809cfa5b5ff2b882948a934cb7.svg
paper_authors.fullname: Omer Dahary
paper_authors.type: user
paper_authors.user: omer11a
paper_authors._id: 66024830c618afa8d796e49d
paper_authors.name: Or Patashnik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-26T09:21:41.319Z
paper_authors._id: 62853516e483e0d37b354ce1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62853516e483e0d37b354ce1/t5Tyd3E07w26B9Z3XpZWI.jpeg
paper_authors.fullname: Or Patashnik
paper_authors.type: user
paper_authors.user: orpatashnik
paper_authors._id: 66024830c618afa8d796e49e
paper_authors.name: Kfir Aberman
paper_authors._id: 66024830c618afa8d796e49f
paper_authors.name: Daniel Cohen-Or
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-26T09:21:53.246Z
paper_authors._id: 628507161949ebcae8e24ec3
paper_authors.avatarUrl: /avatars/008ecb3daa4c8187b5f339f1176b3c39.svg
paper_authors.fullname: Daniel Cohen-Or
paper_authors.type: user
paper_authors.user: cohenor
paper_title: Be Yourself: Bounded Attention for Multi-Subject Text-to-Image
    Generation
paper_summary: Text-to-image diffusion models have an unprecedented ability to generate
  diverse and high-quality images. However, they often struggle to faithfully
  capture the intended semantics of complex input prompts that include multiple
  subjects. Recently, numerous layout-to-image extensions have been introduced to
  improve user control, aiming to localize subjects represented by specific
  tokens. Yet, these methods often produce semantically inaccurate images,
  especially when dealing with multiple semantically or visually similar
  subjects. In this work, we study and analyze the causes of these limitations.
  Our exploration reveals that the primary issue stems from inadvertent semantic
  leakage between subjects in the denoising process. This leakage is attributed
  to the diffusion model's attention layers, which tend to blend the visual
  features of different subjects. To address these issues, we introduce Bounded
  Attention, a training-free method for bounding the information flow in the
  sampling process. Bounded Attention prevents detrimental leakage among subjects
  and enables guiding the generation to promote each subject's individuality,
  even with complex multi-subject conditioning. Through extensive
  experimentation, we demonstrate that our method empowers the generation of
  multiple subjects that better align with given prompts and layouts.
paper_ai_summary: Bounded Attention addresses semantic leakage in text-to-image diffusion models by restricting information flow between subjects, improving the alignment with given prompts and layouts.
paper_ai_keywords: text-to-image diffusion models layout-to-image extensions denoising process attention layers semantic leakage Bounded Attention sampling process multi-subject conditioning