Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse
  Autoencoders
Sparse autoencoders (SAEs) are a promising unsupervised approach for
identifying causally relevant and interpretable linear features in a language
model's (LM) activations. To be useful for downstream tasks, SAEs need to
decompose LM activations faithfully; yet to be interpretable the decomposition
must be sparse -- two objectives that are in tension. In this paper, we
introduce JumpReLU SAEs, which achieve state-of-the-art reconstruction fidelity
at a given sparsity level on Gemma 2 9B activations, compared to other recent
advances such as Gated and TopK SAEs. We also show that this improvement does
not come at the cost of interpretability through manual and automated
interpretability studies. JumpReLU SAEs are a simple modification of vanilla
(ReLU) SAEs -- where we replace the ReLU with a discontinuous JumpReLU
activation function -- and are similarly efficient to train and run. By
utilising straight-through-estimators (STEs) in a principled manner, we show
how it is possible to train JumpReLU SAEs effectively despite the discontinuous
JumpReLU function introduced in the SAE's forward pass. Similarly, we use STEs
to directly train L0 to be sparse, instead of training on proxies such as L1,
avoiding problems like shrinkage.
669dc082a8b62d0515902f8b
Senthooran Rajamanoharan
claimed_verified
2024-07-22T10:05:01.064Z
6477122c99a5ce743ccf2f55
/avatars/dee617374fc609e07eba3bcb2cd16810.svg
Senthooran Rajamanoharan
user
srdm
669dc082a8b62d0515902f8c
Tom Lieberum
claimed_verified
2024-07-22T11:36:28.359Z
632cb97554e2c512c8f7bb31
/avatars/634be7f8a59b6cf639e9e6c5e7ca1fbd.svg
TomL
user
Aric
669dc082a8b62d0515902f8d
Nicolas Sonnerat
669dc082a8b62d0515902f8e
Arthur Conmy
admin_assigned
2024-07-22T08:52:46.744Z
631ecc58daa9591e522e1494
/avatars/4f808fae966e808105e89712c97d90d2.svg
VConm
user
ArthurConmy
669dc082a8b62d0515902f8f
Vikrant Varma
admin_assigned
2024-07-22T08:52:40.931Z
667f25c67a2adf8ac1c940b2
/avatars/ccf62e2e4913d9a8615613f7bb0bfbe2.svg
Vikrant Varma
user
vikrantvarma
669dc082a8b62d0515902f90
János Kramár
admin_assigned
2024-07-22T08:52:59.268Z
6647488b45ee22196e798ba6
/avatars/7c068c30dcd5f07e4d896fd1dce5be2c.svg
Janos Kramar
user
jkramar
669dc082a8b62d0515902f91
Neel Nanda
extracted_pending
2024-07-22T02:14:26.915Z
66918e43a9cf6ccd89d69bf8
/avatars/557d5befd01e8a5bfd4b1b7afd285203.svg
Neel Nanda
user
NeelNanda2
Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse
  Autoencoders
Sparse autoencoders (SAEs) are a promising unsupervised approach for
identifying causally relevant and interpretable linear features in a language
model's (LM) activations. To be useful for downstream tasks, SAEs need to
decompose LM activations faithfully; yet to be interpretable the decomposition
must be sparse -- two objectives that are in tension. In this paper, we
introduce JumpReLU SAEs, which achieve state-of-the-art reconstruction fidelity
at a given sparsity level on Gemma 2 9B activations, compared to other recent
advances such as Gated and TopK SAEs. We also show that this improvement does
not come at the cost of interpretability through manual and automated
interpretability studies. JumpReLU SAEs are a simple modification of vanilla
(ReLU) SAEs -- where we replace the ReLU with a discontinuous JumpReLU
activation function -- and are similarly efficient to train and run. By
utilising straight-through-estimators (STEs) in a principled manner, we show
how it is possible to train JumpReLU SAEs effectively despite the discontinuous
JumpReLU function introduced in the SAE's forward pass. Similarly, we use STEs
to directly train L0 to be sparse, instead of training on proxies such as L1,
avoiding problems like shrinkage.
JumpReLU SAEs achieve high reconstruction fidelity and interpretability in language model activations by using a discontinuous activation function and straight-through-estimators.
Sparse autoencoders JumpReLU Gemma 2 9B Gated SAEs TopK SAEs interpretability straight-through-estimators L0 regularization