title: Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse
    Autoencoders
summary: Sparse autoencoders (SAEs) are a promising unsupervised approach for
  identifying causally relevant and interpretable linear features in a language
  model's (LM) activations. To be useful for downstream tasks, SAEs need to
  decompose LM activations faithfully; yet to be interpretable the decomposition
  must be sparse -- two objectives that are in tension. In this paper, we
  introduce JumpReLU SAEs, which achieve state-of-the-art reconstruction fidelity
  at a given sparsity level on Gemma 2 9B activations, compared to other recent
  advances such as Gated and TopK SAEs. We also show that this improvement does
  not come at the cost of interpretability through manual and automated
  interpretability studies. JumpReLU SAEs are a simple modification of vanilla
  (ReLU) SAEs -- where we replace the ReLU with a discontinuous JumpReLU
  activation function -- and are similarly efficient to train and run. By
  utilising straight-through-estimators (STEs) in a principled manner, we show
  how it is possible to train JumpReLU SAEs effectively despite the discontinuous
  JumpReLU function introduced in the SAE's forward pass. Similarly, we use STEs
  to directly train L0 to be sparse, instead of training on proxies such as L1,
  avoiding problems like shrinkage.
paper_authors._id: 669dc082a8b62d0515902f8b
paper_authors.name: Senthooran Rajamanoharan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-22T10:05:01.064Z
paper_authors._id: 6477122c99a5ce743ccf2f55
paper_authors.avatarUrl: /avatars/dee617374fc609e07eba3bcb2cd16810.svg
paper_authors.fullname: Senthooran Rajamanoharan
paper_authors.type: user
paper_authors.user: srdm
paper_authors._id: 669dc082a8b62d0515902f8c
paper_authors.name: Tom Lieberum
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-22T11:36:28.359Z
paper_authors._id: 632cb97554e2c512c8f7bb31
paper_authors.avatarUrl: /avatars/634be7f8a59b6cf639e9e6c5e7ca1fbd.svg
paper_authors.fullname: TomL
paper_authors.type: user
paper_authors.user: Aric
paper_authors._id: 669dc082a8b62d0515902f8d
paper_authors.name: Nicolas Sonnerat
paper_authors._id: 669dc082a8b62d0515902f8e
paper_authors.name: Arthur Conmy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-22T08:52:46.744Z
paper_authors._id: 631ecc58daa9591e522e1494
paper_authors.avatarUrl: /avatars/4f808fae966e808105e89712c97d90d2.svg
paper_authors.fullname: VConm
paper_authors.type: user
paper_authors.user: ArthurConmy
paper_authors._id: 669dc082a8b62d0515902f8f
paper_authors.name: Vikrant Varma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-22T08:52:40.931Z
paper_authors._id: 667f25c67a2adf8ac1c940b2
paper_authors.avatarUrl: /avatars/ccf62e2e4913d9a8615613f7bb0bfbe2.svg
paper_authors.fullname: Vikrant Varma
paper_authors.type: user
paper_authors.user: vikrantvarma
paper_authors._id: 669dc082a8b62d0515902f90
paper_authors.name: János Kramár
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-22T08:52:59.268Z
paper_authors._id: 6647488b45ee22196e798ba6
paper_authors.avatarUrl: /avatars/7c068c30dcd5f07e4d896fd1dce5be2c.svg
paper_authors.fullname: Janos Kramar
paper_authors.type: user
paper_authors.user: jkramar
paper_authors._id: 669dc082a8b62d0515902f91
paper_authors.name: Neel Nanda
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-22T02:14:26.915Z
paper_authors._id: 66918e43a9cf6ccd89d69bf8
paper_authors.avatarUrl: /avatars/557d5befd01e8a5bfd4b1b7afd285203.svg
paper_authors.fullname: Neel Nanda
paper_authors.type: user
paper_authors.user: NeelNanda2
paper_title: Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse
    Autoencoders
paper_summary: Sparse autoencoders (SAEs) are a promising unsupervised approach for
  identifying causally relevant and interpretable linear features in a language
  model's (LM) activations. To be useful for downstream tasks, SAEs need to
  decompose LM activations faithfully; yet to be interpretable the decomposition
  must be sparse -- two objectives that are in tension. In this paper, we
  introduce JumpReLU SAEs, which achieve state-of-the-art reconstruction fidelity
  at a given sparsity level on Gemma 2 9B activations, compared to other recent
  advances such as Gated and TopK SAEs. We also show that this improvement does
  not come at the cost of interpretability through manual and automated
  interpretability studies. JumpReLU SAEs are a simple modification of vanilla
  (ReLU) SAEs -- where we replace the ReLU with a discontinuous JumpReLU
  activation function -- and are similarly efficient to train and run. By
  utilising straight-through-estimators (STEs) in a principled manner, we show
  how it is possible to train JumpReLU SAEs effectively despite the discontinuous
  JumpReLU function introduced in the SAE's forward pass. Similarly, we use STEs
  to directly train L0 to be sparse, instead of training on proxies such as L1,
  avoiding problems like shrinkage.
paper_ai_summary: JumpReLU SAEs achieve high reconstruction fidelity and interpretability in language model activations by using a discontinuous activation function and straight-through-estimators.
paper_ai_keywords: Sparse autoencoders JumpReLU Gemma 2 9B Gated SAEs TopK SAEs interpretability straight-through-estimators L0 regularization