title: Localizing and Editing Knowledge in Text-to-Image Generative Models
summary: Text-to-Image Diffusion Models such as Stable-Diffusion and Imagen have
  achieved unprecedented quality of photorealism with state-of-the-art FID scores
  on MS-COCO and other generation benchmarks. Given a caption, image generation
  requires fine-grained knowledge about attributes such as object structure,
  style, and viewpoint amongst others. Where does this information reside in
  text-to-image generative models? In our paper, we tackle this question and
  understand how knowledge corresponding to distinct visual attributes is stored
  in large-scale text-to-image diffusion models. We adapt Causal Mediation
  Analysis for text-to-image models and trace knowledge about distinct visual
  attributes to various (causal) components in the (i) UNet and (ii) text-encoder
  of the diffusion model. In particular, we show that unlike generative
  large-language models, knowledge about different attributes is not localized in
  isolated components, but is instead distributed amongst a set of components in
  the conditional UNet. These sets of components are often distinct for different
  visual attributes. Remarkably, we find that the CLIP text-encoder in public
  text-to-image models such as Stable-Diffusion contains only one causal state
  across different visual attributes, and this is the first self-attention layer
  corresponding to the last subject token of the attribute in the caption. This
  is in stark contrast to the causal states in other language models which are
  often the mid-MLP layers. Based on this observation of only one causal state in
  the text-encoder, we introduce a fast, data-free model editing method
  Diff-QuickFix which can effectively edit concepts in text-to-image models.
  DiffQuickFix can edit (ablate) concepts in under a second with a closed-form
  update, providing a significant 1000x speedup and comparable editing
  performance to existing fine-tuning based editing methods.
paper_authors._id: 65372cfde5ad8b46d5391205
paper_authors.name: Samyadeep Basu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-24T08:57:56.743Z
paper_authors._id: 6384d891cafe4e4ac4060206
paper_authors.avatarUrl: /avatars/45eb04bf6e0ed1a4daa14070411b693f.svg
paper_authors.fullname: Samyadeep
paper_authors.type: user
paper_authors.user: samyadeepbasu
paper_authors._id: 65372cfde5ad8b46d5391206
paper_authors.name: Nanxuan Zhao
paper_authors._id: 65372cfde5ad8b46d5391207
paper_authors.name: Vlad Morariu
paper_authors._id: 65372cfde5ad8b46d5391208
paper_authors.name: Soheil Feizi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-24T08:58:14.934Z
paper_authors._id: 64b83f32047fa3db94108a9f
paper_authors.avatarUrl: /avatars/c9afdb5c4c9dc3bf15c9edb223428f5f.svg
paper_authors.fullname: Soheil Feizi
paper_authors.type: user
paper_authors.user: sfeizi
paper_authors._id: 65372cfde5ad8b46d5391209
paper_authors.name: Varun Manjunatha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-24T08:58:20.962Z
paper_authors._id: 64234cafd654afd6931a1b22
paper_authors.avatarUrl: /avatars/d0adeab15e5f0eef08e5d4c91d0e18b0.svg
paper_authors.fullname: Varun Manjunatha
paper_authors.type: user
paper_authors.user: vmanjuna
paper_title: Localizing and Editing Knowledge in Text-to-Image Generative Models
paper_summary: Text-to-Image Diffusion Models such as Stable-Diffusion and Imagen have
  achieved unprecedented quality of photorealism with state-of-the-art FID scores
  on MS-COCO and other generation benchmarks. Given a caption, image generation
  requires fine-grained knowledge about attributes such as object structure,
  style, and viewpoint amongst others. Where does this information reside in
  text-to-image generative models? In our paper, we tackle this question and
  understand how knowledge corresponding to distinct visual attributes is stored
  in large-scale text-to-image diffusion models. We adapt Causal Mediation
  Analysis for text-to-image models and trace knowledge about distinct visual
  attributes to various (causal) components in the (i) UNet and (ii) text-encoder
  of the diffusion model. In particular, we show that unlike generative
  large-language models, knowledge about different attributes is not localized in
  isolated components, but is instead distributed amongst a set of components in
  the conditional UNet. These sets of components are often distinct for different
  visual attributes. Remarkably, we find that the CLIP text-encoder in public
  text-to-image models such as Stable-Diffusion contains only one causal state
  across different visual attributes, and this is the first self-attention layer
  corresponding to the last subject token of the attribute in the caption. This
  is in stark contrast to the causal states in other language models which are
  often the mid-MLP layers. Based on this observation of only one causal state in
  the text-encoder, we introduce a fast, data-free model editing method
  Diff-QuickFix which can effectively edit concepts in text-to-image models.
  DiffQuickFix can edit (ablate) concepts in under a second with a closed-form
  update, providing a significant 1000x speedup and comparable editing
  performance to existing fine-tuning based editing methods.
paper_ai_summary: Text-to-image diffusion models encode visual attributes across multiple components in the UNet and one causal state in the text-encoder, enabling efficient model editing with Diff-QuickFix.
paper_ai_keywords: text-to-image diffusion models Stable-Diffusion Imagen FID scores MS-COCO image generation Causal Mediation Analysis UNet text-encoder generative large-language models self-attention layer attribute editing Diff-QuickFix causal states mid-MLP layers model editing