title: Enhancing Automated Interpretability with Output-Centric Feature
    Descriptions
summary: Automated interpretability pipelines generate natural language descriptions
  for the concepts represented by features in large language models (LLMs), such
  as plants or the first word in a sentence. These descriptions are derived using
  inputs that activate the feature, which may be a dimension or a direction in
  the model's representation space. However, identifying activating inputs is
  costly, and the mechanistic role of a feature in model behavior is determined
  both by how inputs cause a feature to activate and by how feature activation
  affects outputs. Using steering evaluations, we reveal that current pipelines
  provide descriptions that fail to capture the causal effect of the feature on
  outputs. To fix this, we propose efficient, output-centric methods for
  automatically generating feature descriptions. These methods use the tokens
  weighted higher after feature stimulation or the highest weight tokens after
  applying the vocabulary "unembedding" head directly to the feature. Our
  output-centric descriptions better capture the causal effect of a feature on
  model outputs than input-centric descriptions, but combining the two leads to
  the best performance on both input and output evaluations. Lastly, we show that
  output-centric descriptions can be used to find inputs that activate features
  previously thought to be "dead".
paper_authors._id: 67876067c377690c01ab4478
paper_authors.name: Yoav Gur-Arieh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:58:57.951Z
paper_authors._id: 621febb6c7f47c5eb5df001d
paper_authors.avatarUrl: /avatars/6096101dc01f1a7e39b1d0826170412d.svg
paper_authors.fullname: Yoav Gur Arieh
paper_authors.type: user
paper_authors.user: yoavgur
paper_authors._id: 67876067c377690c01ab4479
paper_authors.name: Roy Mayan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:59:28.079Z
paper_authors._id: 66640b931284bd77155ea6cf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66640b931284bd77155ea6cf/pvhrHFLbr0vUPfuYGQT-Q.jpeg
paper_authors.fullname: Roy Mayan
paper_authors.type: user
paper_authors.user: roym44
paper_authors._id: 67876067c377690c01ab447a
paper_authors.name: Chen Agassy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-16T09:00:03.349Z
paper_authors._id: 6788ca64f7306dbc1b757596
paper_authors.avatarUrl: /avatars/5e0cee8de90372fbf8f61aecdc76d498.svg
paper_authors.fullname: Chen Agassy
paper_authors.type: user
paper_authors.user: chenagassy
paper_authors._id: 67876067c377690c01ab447b
paper_authors.name: Atticus Geiger
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-01-15T15:36:25.790Z
paper_authors._id: 627b2d0527dc4650b62eef42
paper_authors.avatarUrl: /avatars/e70381850f5657b54e90f5539f3d74eb.svg
paper_authors.fullname: Atticus Geiger
paper_authors.type: user
paper_authors.user: atticusg
paper_authors._id: 67876067c377690c01ab447c
paper_authors.name: Mor Geva
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:59:15.469Z
paper_authors._id: 610b729f9da682cd54ad9adf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1628140189042-noauth.jpeg
paper_authors.fullname: Mor Geva
paper_authors.type: user
paper_authors.user: mega
paper_title: Enhancing Automated Interpretability with Output-Centric Feature
    Descriptions
paper_summary: Automated interpretability pipelines generate natural language descriptions
  for the concepts represented by features in large language models (LLMs), such
  as plants or the first word in a sentence. These descriptions are derived using
  inputs that activate the feature, which may be a dimension or a direction in
  the model's representation space. However, identifying activating inputs is
  costly, and the mechanistic role of a feature in model behavior is determined
  both by how inputs cause a feature to activate and by how feature activation
  affects outputs. Using steering evaluations, we reveal that current pipelines
  provide descriptions that fail to capture the causal effect of the feature on
  outputs. To fix this, we propose efficient, output-centric methods for
  automatically generating feature descriptions. These methods use the tokens
  weighted higher after feature stimulation or the highest weight tokens after
  applying the vocabulary "unembedding" head directly to the feature. Our
  output-centric descriptions better capture the causal effect of a feature on
  model outputs than input-centric descriptions, but combining the two leads to
  the best performance on both input and output evaluations. Lastly, we show that
  output-centric descriptions can be used to find inputs that activate features
  previously thought to be "dead".
paper_ai_summary: Efficient, output-centric methods improve feature descriptions in large language models by better capturing causal effects and identifying previously inactive features.
paper_ai_keywords: automated interpretability pipelines large language models (LLMs) feature descriptions steering evaluations input-centric descriptions output-centric descriptions feature stimulation vocabulary "unembedding" head dead features