title: Nested Attention: Semantic-aware Attention Values for Concept
    Personalization
summary: Personalizing text-to-image models to generate images of specific subjects
  across diverse scenes and styles is a rapidly advancing field. Current
  approaches often face challenges in maintaining a balance between identity
  preservation and alignment with the input text prompt. Some methods rely on a
  single textual token to represent a subject, which limits expressiveness, while
  others employ richer representations but disrupt the model's prior, diminishing
  prompt alignment. In this work, we introduce Nested Attention, a novel
  mechanism that injects a rich and expressive image representation into the
  model's existing cross-attention layers. Our key idea is to generate
  query-dependent subject values, derived from nested attention layers that learn
  to select relevant subject features for each region in the generated image. We
  integrate these nested layers into an encoder-based personalization method, and
  show that they enable high identity preservation while adhering to input text
  prompts. Our approach is general and can be trained on various domains.
  Additionally, its prior preservation allows us to combine multiple personalized
  subjects from different domains in a single image.
paper_authors._id: 67780d1a007c81c7ace7d132
paper_authors.name: Or Patashnik
paper_authors._id: 67780d1a007c81c7ace7d133
paper_authors.name: Rinon Gal
paper_authors._id: 67780d1a007c81c7ace7d134
paper_authors.name: Daniil Ostashev
paper_authors._id: 67780d1a007c81c7ace7d135
paper_authors.name: Sergey Tulyakov
paper_authors._id: 67780d1a007c81c7ace7d136
paper_authors.name: Kfir Aberman
paper_authors._id: 67780d1a007c81c7ace7d137
paper_authors.name: Daniel Cohen-Or
paper_title: Nested Attention: Semantic-aware Attention Values for Concept
    Personalization
paper_summary: Personalizing text-to-image models to generate images of specific subjects
  across diverse scenes and styles is a rapidly advancing field. Current
  approaches often face challenges in maintaining a balance between identity
  preservation and alignment with the input text prompt. Some methods rely on a
  single textual token to represent a subject, which limits expressiveness, while
  others employ richer representations but disrupt the model's prior, diminishing
  prompt alignment. In this work, we introduce Nested Attention, a novel
  mechanism that injects a rich and expressive image representation into the
  model's existing cross-attention layers. Our key idea is to generate
  query-dependent subject values, derived from nested attention layers that learn
  to select relevant subject features for each region in the generated image. We
  integrate these nested layers into an encoder-based personalization method, and
  show that they enable high identity preservation while adhering to input text
  prompts. Our approach is general and can be trained on various domains.
  Additionally, its prior preservation allows us to combine multiple personalized
  subjects from different domains in a single image.
paper_ai_summary: Nested Attention enhances image-text alignment by injecting expressive subject representations into cross-attention layers, enabling high identity preservation and multi-domain subject combination.
paper_ai_keywords: cross-attention layers Nested Attention query-dependent subject values identity preservation text prompts encoder-based personalization