title: Mechanistic Permutability: Match Features Across Layers
summary: Understanding how features evolve across layers in deep neural networks is a
  fundamental challenge in mechanistic interpretability, particularly due to
  polysemanticity and feature superposition. While Sparse Autoencoders (SAEs)
  have been used to extract interpretable features from individual layers,
  aligning these features across layers has remained an open problem. In this
  paper, we introduce SAE Match, a novel, data-free method for aligning SAE
  features across different layers of a neural network. Our approach involves
  matching features by minimizing the mean squared error between the folded
  parameters of SAEs, a technique that incorporates activation thresholds into
  the encoder and decoder weights to account for differences in feature scales.
  Through extensive experiments on the Gemma 2 language model, we demonstrate
  that our method effectively captures feature evolution across layers, improving
  feature matching quality. We also show that features persist over several
  layers and that our approach can approximate hidden states across layers. Our
  work advances the understanding of feature dynamics in neural networks and
  provides a new tool for mechanistic interpretability studies.
paper_authors._id: 670cd93f9cf5450e15cfe12d
paper_authors.name: Nikita Balagansky
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T10:47:41.728Z
paper_authors._id: 60b364e7f88532cd79eaff7b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654185363389-60b364e7f88532cd79eaff7b.jpeg
paper_authors.fullname: Nikita Balagansky
paper_authors.type: user
paper_authors.user: elephantmipt
paper_authors._id: 670cd93f9cf5450e15cfe12e
paper_authors.name: Ian Maksimov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-15T09:11:15.152Z
paper_authors._id: 65cbcb760b3bd8f5ce402bd5
paper_authors.avatarUrl: /avatars/2a60e344cf219eed58217ccf5db26680.svg
paper_authors.fullname: Ian Maksimov
paper_authors.type: user
paper_authors.user: BayesianMonster
paper_authors._id: 670cd93f9cf5450e15cfe12f
paper_authors.name: Daniil Gavrilov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T08:43:11.835Z
paper_authors._id: 62a9c8edc19f92ae443ab37f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669110208492-62a9c8edc19f92ae443ab37f.png
paper_authors.fullname: Daniil Gavrilov
paper_authors.type: user
paper_authors.user: kefirski
paper_title: Mechanistic Permutability: Match Features Across Layers
paper_summary: Understanding how features evolve across layers in deep neural networks is a
  fundamental challenge in mechanistic interpretability, particularly due to
  polysemanticity and feature superposition. While Sparse Autoencoders (SAEs)
  have been used to extract interpretable features from individual layers,
  aligning these features across layers has remained an open problem. In this
  paper, we introduce SAE Match, a novel, data-free method for aligning SAE
  features across different layers of a neural network. Our approach involves
  matching features by minimizing the mean squared error between the folded
  parameters of SAEs, a technique that incorporates activation thresholds into
  the encoder and decoder weights to account for differences in feature scales.
  Through extensive experiments on the Gemma 2 language model, we demonstrate
  that our method effectively captures feature evolution across layers, improving
  feature matching quality. We also show that features persist over several
  layers and that our approach can approximate hidden states across layers. Our
  work advances the understanding of feature dynamics in neural networks and
  provides a new tool for mechanistic interpretability studies.
paper_ai_keywords: Sparse Autoencoders (SAEs) mechanistic interpretability polysemanticity feature evolution feature superpositions SAE features data-free method mean information error folded parameters activation thresholds encoder and decoder weights feature scales Gemma 2 language model feature matching quality feature dynamics