title: AtP*: An efficient and scalable method for localizing LLM behaviour to
    components
summary: Activation Patching is a method of directly computing causal attributions of
  behavior to model components. However, applying it exhaustively requires a
  sweep with cost scaling linearly in the number of model components, which can
  be prohibitively expensive for SoTA Large Language Models (LLMs). We
  investigate Attribution Patching (AtP), a fast gradient-based approximation to
  Activation Patching and find two classes of failure modes of AtP which lead to
  significant false negatives. We propose a variant of AtP called AtP*, with two
  changes to address these failure modes while retaining scalability. We present
  the first systematic study of AtP and alternative methods for faster activation
  patching and show that AtP significantly outperforms all other investigated
  methods, with AtP* providing further significant improvement. Finally, we
  provide a method to bound the probability of remaining false negatives of AtP*
  estimates.
paper_authors._id: 65e55bd5ad42606f6f3f536c
paper_authors.name: János Kramár
paper_authors._id: 65e55bd5ad42606f6f3f536d
paper_authors.name: Tom Lieberum
paper_authors._id: 65e55bd5ad42606f6f3f536e
paper_authors.name: Rohin Shah
paper_authors._id: 65e55bd5ad42606f6f3f536f
paper_authors.name: Neel Nanda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-04T10:02:23.445Z
paper_authors._id: 62669380c8bc5cf80ca97350
paper_authors.avatarUrl: /avatars/6d5cd2261163308b82341c1ce28984d1.svg
paper_authors.fullname: Neel Nanda
paper_authors.type: user
paper_authors.user: NeelNanda
paper_title: AtP*: An efficient and scalable method for localizing LLM behaviour to
    components
paper_summary: Activation Patching is a method of directly computing causal attributions of
  behavior to model components. However, applying it exhaustively requires a
  sweep with cost scaling linearly in the number of model components, which can
  be prohibitively expensive for SoTA Large Language Models (LLMs). We
  investigate Attribution Patching (AtP), a fast gradient-based approximation to
  Activation Patching and find two classes of failure modes of AtP which lead to
  significant false negatives. We propose a variant of AtP called AtP*, with two
  changes to address these failure modes while retaining scalability. We present
  the first systematic study of AtP and alternative methods for faster activation
  patching and show that AtP significantly outperforms all other investigated
  methods, with AtP* providing further significant improvement. Finally, we
  provide a method to bound the probability of remaining false negatives of AtP*
  estimates.
paper_ai_summary: Attribution Patching (AtP) is a scalable method for approximating causal attributions in large language models, outperforming other techniques including its improved variant AtP*, which also provides a method to bound false negatives.
paper_ai_keywords: Activation Patching Attribution Patching AtP AtP* Large Language Models (LLMs) causal attributions gradient-based approximation false negatives