title: Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
summary: Sparse autoencoders (SAEs) are an unsupervised method for learning a sparse
  decomposition of a neural network's latent representations into seemingly
  interpretable features. Despite recent excitement about their potential,
  research applications outside of industry are limited by the high cost of
  training a comprehensive suite of SAEs. In this work, we introduce Gemma Scope,
  an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2
  2B and 9B and select layers of Gemma 2 27B base models. We primarily train SAEs
  on the Gemma 2 pre-trained models, but additionally release SAEs trained on
  instruction-tuned Gemma 2 9B for comparison. We evaluate the quality of each
  SAE on standard metrics and release these results. We hope that by releasing
  these SAE weights, we can help make more ambitious safety and interpretability
  research easier for the community. Weights and a tutorial can be found at
  https://huggingface.co/google/gemma-scope and an interactive demo can be found
  at https://www.neuronpedia.org/gemma-scope
paper_authors._id: 66b97edaa0ebe4b8a50f1b44
paper_authors.name: Tom Lieberum
paper_authors._id: 66b97edaa0ebe4b8a50f1b45
paper_authors.name: Senthooran Rajamanoharan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T09:58:33.347Z
paper_authors._id: 6477122c99a5ce743ccf2f55
paper_authors.avatarUrl: /avatars/dee617374fc609e07eba3bcb2cd16810.svg
paper_authors.fullname: Senthooran Rajamanoharan
paper_authors.type: user
paper_authors.user: srdm
paper_authors._id: 66b97edaa0ebe4b8a50f1b46
paper_authors.name: Arthur Conmy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-13T07:38:33.046Z
paper_authors._id: 631ecc58daa9591e522e1494
paper_authors.avatarUrl: /avatars/4f808fae966e808105e89712c97d90d2.svg
paper_authors.fullname: VConm
paper_authors.type: user
paper_authors.user: ArthurConmy
paper_authors._id: 66b97edaa0ebe4b8a50f1b47
paper_authors.name: Lewis Smith
paper_authors._id: 66b97edaa0ebe4b8a50f1b48
paper_authors.name: Nicolas Sonnerat
paper_authors._id: 66b97edaa0ebe4b8a50f1b49
paper_authors.name: Vikrant Varma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T09:59:12.295Z
paper_authors._id: 667f25c67a2adf8ac1c940b2
paper_authors.avatarUrl: /avatars/ccf62e2e4913d9a8615613f7bb0bfbe2.svg
paper_authors.fullname: Vikrant Varma
paper_authors.type: user
paper_authors.user: vikrantvarma
paper_authors._id: 66b97edaa0ebe4b8a50f1b4a
paper_authors.name: János Kramár
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T09:59:22.691Z
paper_authors._id: 6647488b45ee22196e798ba6
paper_authors.avatarUrl: /avatars/7c068c30dcd5f07e4d896fd1dce5be2c.svg
paper_authors.fullname: Janos Kramar
paper_authors.type: user
paper_authors.user: jkramar
paper_authors._id: 66b97edaa0ebe4b8a50f1b4b
paper_authors.name: Anca Dragan
paper_authors._id: 66b97edaa0ebe4b8a50f1b4c
paper_authors.name: Rohin Shah
paper_authors._id: 66b97edaa0ebe4b8a50f1b4d
paper_authors.name: Neel Nanda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T10:00:03.588Z
paper_authors._id: 62669380c8bc5cf80ca97350
paper_authors.avatarUrl: /avatars/6d5cd2261163308b82341c1ce28984d1.svg
paper_authors.fullname: Neel Nanda
paper_authors.type: user
paper_authors.user: NeelNanda
paper_title: Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
paper_summary: Sparse autoencoders (SAEs) are an unsupervised method for learning a sparse
  decomposition of a neural network's latent representations into seemingly
  interpretable features. Despite recent excitement about their potential,
  research applications outside of industry are limited by the high cost of
  training a comprehensive suite of SAEs. In this work, we introduce Gemma Scope,
  an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2
  2B and 9B and select layers of Gemma 2 27B base models. We primarily train SAEs
  on the Gemma 2 pre-trained models, but additionally release SAEs trained on
  instruction-tuned Gemma 2 9B for comparison. We evaluate the quality of each
  SAE on standard metrics and release these results. We hope that by releasing
  these SAE weights, we can help make more ambitious safety and interpretability
  research easier for the community. Weights and a tutorial can be found at
  https://huggingface.co/google/gemma-scope and an interactive demo can be found
  at https://www.neuronpedia.org/gemma-scope
paper_ai_keywords: sparse autoencoders unsupervised method sparse decomposition latent representations JumpReLU Gemma 2 instruction-tuned SAE weights safety research interpretability research