AtP*: An efficient and scalable method for localizing LLM behaviour to
  components
Activation Patching is a method of directly computing causal attributions of
behavior to model components. However, applying it exhaustively requires a
sweep with cost scaling linearly in the number of model components, which can
be prohibitively expensive for SoTA Large Language Models (LLMs). We
investigate Attribution Patching (AtP), a fast gradient-based approximation to
Activation Patching and find two classes of failure modes of AtP which lead to
significant false negatives. We propose a variant of AtP called AtP*, with two
changes to address these failure modes while retaining scalability. We present
the first systematic study of AtP and alternative methods for faster activation
patching and show that AtP significantly outperforms all other investigated
methods, with AtP* providing further significant improvement. Finally, we
provide a method to bound the probability of remaining false negatives of AtP*
estimates.
65e55bd5ad42606f6f3f536c
János Kramár
65e55bd5ad42606f6f3f536d
Tom Lieberum
65e55bd5ad42606f6f3f536e
Rohin Shah
65e55bd5ad42606f6f3f536f
Neel Nanda
admin_assigned
2024-03-04T10:02:23.445Z
62669380c8bc5cf80ca97350
/avatars/6d5cd2261163308b82341c1ce28984d1.svg
Neel Nanda
user
NeelNanda
AtP*: An efficient and scalable method for localizing LLM behaviour to
  components
Activation Patching is a method of directly computing causal attributions of
behavior to model components. However, applying it exhaustively requires a
sweep with cost scaling linearly in the number of model components, which can
be prohibitively expensive for SoTA Large Language Models (LLMs). We
investigate Attribution Patching (AtP), a fast gradient-based approximation to
Activation Patching and find two classes of failure modes of AtP which lead to
significant false negatives. We propose a variant of AtP called AtP*, with two
changes to address these failure modes while retaining scalability. We present
the first systematic study of AtP and alternative methods for faster activation
patching and show that AtP significantly outperforms all other investigated
methods, with AtP* providing further significant improvement. Finally, we
provide a method to bound the probability of remaining false negatives of AtP*
estimates.
Attribution Patching (AtP) is a scalable method for approximating causal attributions in large language models, outperforming other techniques including its improved variant AtP*, which also provides a method to bound false negatives.
Activation Patching Attribution Patching AtP AtP* Large Language Models (LLMs) causal attributions gradient-based approximation false negatives