title: Silent Branding Attack: Trigger-free Data Poisoning Attack on
    Text-to-Image Diffusion Models
summary: Text-to-image diffusion models have achieved remarkable success in generating
  high-quality contents from text prompts. However, their reliance on publicly
  available data and the growing trend of data sharing for fine-tuning make these
  models particularly vulnerable to data poisoning attacks. In this work, we
  introduce the Silent Branding Attack, a novel data poisoning method that
  manipulates text-to-image diffusion models to generate images containing
  specific brand logos or symbols without any text triggers. We find that when
  certain visual patterns are repeatedly in the training data, the model learns
  to reproduce them naturally in its outputs, even without prompt mentions.
  Leveraging this, we develop an automated data poisoning algorithm that
  unobtrusively injects logos into original images, ensuring they blend naturally
  and remain undetected. Models trained on this poisoned dataset generate images
  containing logos without degrading image quality or text alignment. We
  experimentally validate our silent branding attack across two realistic
  settings on large-scale high-quality image datasets and style personalization
  datasets, achieving high success rates even without a specific text trigger.
  Human evaluation and quantitative metrics including logo detection show that
  our method can stealthily embed logos.
paper_authors._id: 67d37754e07f664c7325f236
paper_authors.name: Sangwon Jang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:56:49.038Z
paper_authors._id: 63bbf972d8d676a2299cdb44
paper_authors.avatarUrl: /avatars/cd038f11dc1007b1267324b34c165dda.svg
paper_authors.fullname: Sangwon
paper_authors.type: user
paper_authors.user: agwmon
paper_authors._id: 67d37754e07f664c7325f237
paper_authors.name: June Suk Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:05:59.636Z
paper_authors._id: 66c6edcc91dced946471bc13
paper_authors.avatarUrl: /avatars/55cc8593da6540e1566e1de9d7133f9f.svg
paper_authors.fullname: June Suk Choi
paper_authors.type: user
paper_authors.user: wchoi403
paper_authors._id: 67d37754e07f664c7325f238
paper_authors.name: Jaehyeong Jo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:06:19.773Z
paper_authors._id: 65e5bd4568234ef5d6decadc
paper_authors.avatarUrl: /avatars/c41095a946c0176b949c0b3566136c05.svg
paper_authors.fullname: Jaehyeong Jo
paper_authors.type: user
paper_authors.user: harryjo97
paper_authors._id: 67d37754e07f664c7325f239
paper_authors.name: Kimin Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:06:26.649Z
paper_authors._id: 635097ec59bfa9a85d4207b2
paper_authors.avatarUrl: /avatars/787085894e9e6538b6b3e3051efe9eea.svg
paper_authors.fullname: Kimin Lee
paper_authors.type: user
paper_authors.user: kiminle2
paper_authors._id: 67d37754e07f664c7325f23a
paper_authors.name: Sung Ju Hwang
paper_title: Silent Branding Attack: Trigger-free Data Poisoning Attack on
    Text-to-Image Diffusion Models
paper_summary: Text-to-image diffusion models have achieved remarkable success in generating
  high-quality contents from text prompts. However, their reliance on publicly
  available data and the growing trend of data sharing for fine-tuning make these
  models particularly vulnerable to data poisoning attacks. In this work, we
  introduce the Silent Branding Attack, a novel data poisoning method that
  manipulates text-to-image diffusion models to generate images containing
  specific brand logos or symbols without any text triggers. We find that when
  certain visual patterns are repeatedly in the training data, the model learns
  to reproduce them naturally in its outputs, even without prompt mentions.
  Leveraging this, we develop an automated data poisoning algorithm that
  unobtrusively injects logos into original images, ensuring they blend naturally
  and remain undetected. Models trained on this poisoned dataset generate images
  containing logos without degrading image quality or text alignment. We
  experimentally validate our silent branding attack across two realistic
  settings on large-scale high-quality image datasets and style personalization
  datasets, achieving high success rates even without a specific text trigger.
  Human evaluation and quantitative metrics including logo detection show that
  our method can stealthily embed logos.
paper_ai_keywords: text-to-image diffusion models high-quality contents text prompts data poisoning attacks Silent Branding Attack visual patterns data poisoning algorithm logos style personalization datasets logo detection