title: Sparse Autoencoders Find Highly Interpretable Features in Language
    Models
summary: One of the roadblocks to a better understanding of neural networks' internals
  is polysemanticity, where neurons appear to activate in multiple,
  semantically distinct contexts. Polysemanticity prevents us from identifying
  concise, human-understandable explanations for what neural networks are doing
  internally. One hypothesised cause of polysemanticity is
  superposition, where neural networks represent more features than they
  have neurons by assigning features to an overcomplete set of directions in
  activation space, rather than to individual neurons. Here, we attempt to
  identify those directions, using sparse autoencoders to reconstruct the
  internal activations of a language model. These autoencoders learn sets of
  sparsely activating features that are more interpretable and monosemantic than
  directions identified by alternative approaches, where interpretability is
  measured by automated methods. Ablating these features enables precise model
  editing, for example, by removing capabilities such as pronoun prediction,
  while disrupting model behaviour less than prior techniques. This work
  indicates that it is possible to resolve superposition in language models using
  a scalable, unsupervised method. Our method may serve as a foundation for
  future mechanistic interpretability work, which we hope will enable greater
  model transparency and steerability.
paper_authors._id: 6507b936a7ba30bf62ccc74f
paper_authors.name: Hoagy Cunningham
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-18T10:18:29.959Z
paper_authors._id: 636146e7472131c3bc538bd8
paper_authors.avatarUrl: /avatars/9db880163cc0eea796165d8bf5e2a91f.svg
paper_authors.fullname: Hoagy Cunningham
paper_authors.type: user
paper_authors.user: HoagyC
paper_authors._id: 6507b936a7ba30bf62ccc750
paper_authors.name: Aidan Ewart
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-18T10:18:38.015Z
paper_authors._id: 643bbb729f5d314db2d89c22
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643bbb729f5d314db2d89c22/bWxfTXlWY1y1-rS7Czqup.jpeg
paper_authors.fullname: Aidan Ewart
paper_authors.type: user
paper_authors.user: Baidicoot
paper_authors._id: 6507b936a7ba30bf62ccc751
paper_authors.name: Logan Riggs
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-18T10:18:44.781Z
paper_authors._id: 6366a684953957183c0fb596
paper_authors.avatarUrl: /avatars/1a45039f224d78f126689a784f3f05f4.svg
paper_authors.fullname: Logan Riggs Smith
paper_authors.type: user
paper_authors.user: Elriggs
paper_authors._id: 6507b936a7ba30bf62ccc752
paper_authors.name: Robert Huben
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-18T10:18:53.618Z
paper_authors._id: 647e0050a49bffab5d7108c2
paper_authors.avatarUrl: /avatars/707dbd12cc9d4b4711fd7e660385cd2d.svg
paper_authors.fullname: Robert Huben
paper_authors.type: user
paper_authors.user: RobertHuben
paper_authors._id: 6507b936a7ba30bf62ccc753
paper_authors.name: Lee Sharkey
paper_title: Sparse Autoencoders Find Highly Interpretable Features in Language
    Models
paper_summary: One of the roadblocks to a better understanding of neural networks' internals
  is polysemanticity, where neurons appear to activate in multiple,
  semantically distinct contexts. Polysemanticity prevents us from identifying
  concise, human-understandable explanations for what neural networks are doing
  internally. One hypothesised cause of polysemanticity is
  superposition, where neural networks represent more features than they
  have neurons by assigning features to an overcomplete set of directions in
  activation space, rather than to individual neurons. Here, we attempt to
  identify those directions, using sparse autoencoders to reconstruct the
  internal activations of a language model. These autoencoders learn sets of
  sparsely activating features that are more interpretable and monosemantic than
  directions identified by alternative approaches, where interpretability is
  measured by automated methods. Ablating these features enables precise model
  editing, for example, by removing capabilities such as pronoun prediction,
  while disrupting model behaviour less than prior techniques. This work
  indicates that it is possible to resolve superposition in language models using
  a scalable, unsupervised method. Our method may serve as a foundation for
  future mechanistic interpretability work, which we hope will enable greater
  model transparency and steerability.
paper_ai_keywords: polysemanticity superposition sparse autoencoders interpretability monosemantic model editing pronoun prediction mechanistic interpretability