title: Concept Steerers: Leveraging K-Sparse Autoencoders for Controllable
    Generations
summary: Despite the remarkable progress in text-to-image generative models, they are
  prone to adversarial attacks and inadvertently generate unsafe, unethical
  content. Existing approaches often rely on fine-tuning models to remove
  specific concepts, which is computationally expensive, lack scalability, and/or
  compromise generation quality. In this work, we propose a novel framework
  leveraging k-sparse autoencoders (k-SAEs) to enable efficient and interpretable
  concept manipulation in diffusion models. Specifically, we first identify
  interpretable monosemantic concepts in the latent space of text embeddings and
  leverage them to precisely steer the generation away or towards a given concept
  (e.g., nudity) or to introduce a new concept (e.g., photographic style).
  Through extensive experiments, we demonstrate that our approach is very simple,
  requires no retraining of the base model nor LoRA adapters, does not compromise
  the generation quality, and is robust to adversarial prompt manipulations. Our
  method yields an improvement of 20.01% in unsafe concept removal,
  is effective in style manipulation, and is sim5x faster than
  current state-of-the-art.
paper_authors._id: 67a0f59c5685d37e28880943
paper_authors.name: Dahye Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-03T16:59:53.099Z
paper_authors._id: 64bcc06fb567ae97c3272d3d
paper_authors.avatarUrl: /avatars/bcb61fe9e575154d84913a1501971f1a.svg
paper_authors.fullname: kim
paper_authors.type: user
paper_authors.user: dahyekim
paper_authors._id: 67a0f59c5685d37e28880944
paper_authors.name: Deepti Ghadiyaram
paper_title: Concept Steerers: Leveraging K-Sparse Autoencoders for Controllable
    Generations
paper_summary: Despite the remarkable progress in text-to-image generative models, they are
  prone to adversarial attacks and inadvertently generate unsafe, unethical
  content. Existing approaches often rely on fine-tuning models to remove
  specific concepts, which is computationally expensive, lack scalability, and/or
  compromise generation quality. In this work, we propose a novel framework
  leveraging k-sparse autoencoders (k-SAEs) to enable efficient and interpretable
  concept manipulation in diffusion models. Specifically, we first identify
  interpretable monosemantic concepts in the latent space of text embeddings and
  leverage them to precisely steer the generation away or towards a given concept
  (e.g., nudity) or to introduce a new concept (e.g., photographic style).
  Through extensive experiments, we demonstrate that our approach is very simple,
  requires no retraining of the base model nor LoRA adapters, does not compromise
  the generation quality, and is robust to adversarial prompt manipulations. Our
  method yields an improvement of 20.01% in unsafe concept removal,
  is effective in style manipulation, and is sim5x faster than
  current state-of-the-art.
paper_ai_summary: The framework using k-sparse autoencoders in diffusion models enables efficient and interpretable concept manipulation, improving unsafe concept removal and style manipulation without retraining.
paper_ai_keywords: k-sparse autoencoders k-SAEs diffusion models latent space text embeddings monosemantic concepts prompt manipulations