title: SAeUron: Interpretable Concept Unlearning in Diffusion Models with
    Sparse Autoencoders
summary: Diffusion models, while powerful, can inadvertently generate harmful or
  undesirable content, raising significant ethical and safety concerns. Recent
  machine unlearning approaches offer potential solutions but often lack
  transparency, making it difficult to understand the changes they introduce to
  the base model. In this work, we introduce SAeUron, a novel method leveraging
  features learned by sparse autoencoders (SAEs) to remove unwanted concepts in
  text-to-image diffusion models. First, we demonstrate that SAEs, trained in an
  unsupervised manner on activations from multiple denoising timesteps of the
  diffusion model, capture sparse and interpretable features corresponding to
  specific concepts. Building on this, we propose a feature selection method that
  enables precise interventions on model activations to block targeted content
  while preserving overall performance. Evaluation with the competitive
  UnlearnCanvas benchmark on object and style unlearning highlights SAeUron's
  state-of-the-art performance. Moreover, we show that with a single SAE, we can
  remove multiple concepts simultaneously and that in contrast to other methods,
  SAeUron mitigates the possibility of generating unwanted content, even under
  adversarial attack. Code and checkpoints are available at:
  https://github.com/cywinski/SAeUron.
paper_authors._id: 67a07bbb8e344720ae1a6008
paper_authors.name: Bartosz Cywiński
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-03T11:07:48.609Z
paper_authors._id: 6422f416a73327caad9d1d86
paper_authors.avatarUrl: /avatars/aa3639277cd1732504402fc64a57eff8.svg
paper_authors.fullname: Bartosz Cywiński
paper_authors.type: user
paper_authors.user: bcywinski
paper_authors._id: 67a07bbb8e344720ae1a6009
paper_authors.name: Kamil Deja
paper_title: SAeUron: Interpretable Concept Unlearning in Diffusion Models with
    Sparse Autoencoders
paper_summary: Diffusion models, while powerful, can inadvertently generate harmful or
  undesirable content, raising significant ethical and safety concerns. Recent
  machine unlearning approaches offer potential solutions but often lack
  transparency, making it difficult to understand the changes they introduce to
  the base model. In this work, we introduce SAeUron, a novel method leveraging
  features learned by sparse autoencoders (SAEs) to remove unwanted concepts in
  text-to-image diffusion models. First, we demonstrate that SAEs, trained in an
  unsupervised manner on activations from multiple denoising timesteps of the
  diffusion model, capture sparse and interpretable features corresponding to
  specific concepts. Building on this, we propose a feature selection method that
  enables precise interventions on model activations to block targeted content
  while preserving overall performance. Evaluation with the competitive
  UnlearnCanvas benchmark on object and style unlearning highlights SAeUron's
  state-of-the-art performance. Moreover, we show that with a single SAE, we can
  remove multiple concepts simultaneously and that in contrast to other methods,
  SAeUron mitigates the possibility of generating unwanted content, even under
  adversarial attack. Code and checkpoints are available at:
  https://github.com/cywinski/SAeUron.
paper_ai_summary: SAeUron uses sparse autoencoders to remove unwanted concepts from diffusion models in text-to-image tasks, improving safety and performance.
paper_ai_keywords: diffusion models sparse autoencoders feature selection UnlearnCanvas activation denoising adversarial attack