title: SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder
    Guardrails for Precision Unlearning in LLMs
summary: Machine unlearning is a promising approach to improve LLM safety by removing
  unwanted knowledge from the model. However, prevailing gradient-based
  unlearning methods suffer from issues such as high computational costs,
  hyperparameter instability, poor sequential unlearning capability,
  vulnerability to relearning attacks, low data efficiency, and lack of
  interpretability. While Sparse Autoencoders are well-suited to improve these
  aspects by enabling targeted activation-based unlearning, prior approaches
  underperform gradient-based methods. This work demonstrates that, contrary to
  these earlier findings, SAEs can significantly improve unlearning when employed
  dynamically. We introduce Dynamic DAE Guardrails (DSG), a novel
  method for precision unlearning that leverages principled feature selection and
  a dynamic classifier. Our experiments show DSG substantially outperforms
  leading unlearning methods, achieving superior forget-utility trade-offs. DSG
  addresses key drawbacks of gradient-based approaches for unlearning -- offering
  enhanced computational efficiency and stability, robust performance in
  sequential unlearning, stronger resistance to relearning attacks, better data
  efficiency including zero-shot settings, and more interpretable unlearning.
paper_authors._id: 67fcb3584a92187863e732d5
paper_authors.name: Aashiq Muhamed
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:55:50.625Z
paper_authors._id: 64755a83e0b188d3cb2579d8
paper_authors.avatarUrl: /avatars/2c50590905f4bd398a4c9991e1b4b5bb.svg
paper_authors.fullname: Aashiq Muhamed
paper_authors.type: user
paper_authors.user: aashiqmuhamed
paper_authors._id: 67fcb3584a92187863e732d6
paper_authors.name: Jacopo Bonato
paper_authors._id: 67fcb3584a92187863e732d7
paper_authors.name: Mona Diab
paper_authors._id: 67fcb3584a92187863e732d8
paper_authors.name: Virginia Smith
paper_title: SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder
    Guardrails for Precision Unlearning in LLMs
paper_summary: Machine unlearning is a promising approach to improve LLM safety by removing
  unwanted knowledge from the model. However, prevailing gradient-based
  unlearning methods suffer from issues such as high computational costs,
  hyperparameter instability, poor sequential unlearning capability,
  vulnerability to relearning attacks, low data efficiency, and lack of
  interpretability. While Sparse Autoencoders are well-suited to improve these
  aspects by enabling targeted activation-based unlearning, prior approaches
  underperform gradient-based methods. This work demonstrates that, contrary to
  these earlier findings, SAEs can significantly improve unlearning when employed
  dynamically. We introduce Dynamic DAE Guardrails (DSG), a novel
  method for precision unlearning that leverages principled feature selection and
  a dynamic classifier. Our experiments show DSG substantially outperforms
  leading unlearning methods, achieving superior forget-utility trade-offs. DSG
  addresses key drawbacks of gradient-based approaches for unlearning -- offering
  enhanced computational efficiency and stability, robust performance in
  sequential unlearning, stronger resistance to relearning attacks, better data
  efficiency including zero-shot settings, and more interpretable unlearning.
paper_ai_summary: Dynamic DAE Guardrails improve machine unlearning in large language models by overcoming limitations of gradient-based methods and offering enhanced efficiency, stability, and interpretability.
paper_ai_keywords: Sparse Autoencoders Dynamic DAE Guardrails precision unlearning feature selection dynamic classifier forget-utility trade-offs sequential unlearning relearning attacks data efficiency zero-shot settings interpretability