title: Analyze Feature Flow to Enhance Interpretation and Steering in Language
    Models
summary: We introduce a new approach to systematically map features discovered by
  sparse autoencoder across consecutive layers of large language models,
  extending earlier work that examined inter-layer feature links. By using a
  data-free cosine similarity technique, we trace how specific features persist,
  transform, or first appear at each stage. This method yields granular flow
  graphs of feature evolution, enabling fine-grained interpretability and
  mechanistic insights into model computations. Crucially, we demonstrate how
  these cross-layer feature maps facilitate direct steering of model behavior by
  amplifying or suppressing chosen features, achieving targeted thematic control
  in text generation. Together, our findings highlight the utility of a causal,
  cross-layer interpretability framework that not only clarifies how features
  develop through forward passes but also provides new means for transparent
  manipulation of large language models.
paper_authors._id: 67a59c4e7ffacd843a56404a
paper_authors.name: Daniil Laptev
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-07T09:58:04.546Z
paper_authors._id: 634c5f8cfb80cc6bcaf42c03
paper_authors.avatarUrl: /avatars/1f37db0e70cbaf9707f4c8cbcee37ca0.svg
paper_authors.fullname: Daniil Laptev
paper_authors.type: user
paper_authors.user: dlaptev
paper_authors._id: 67a59c4e7ffacd843a56404b
paper_authors.name: Nikita Balagansky
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-07T09:58:02.693Z
paper_authors._id: 60b364e7f88532cd79eaff7b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654185363389-60b364e7f88532cd79eaff7b.jpeg
paper_authors.fullname: Nikita Balagansky
paper_authors.type: user
paper_authors.user: elephantmipt
paper_authors._id: 67a59c4e7ffacd843a56404c
paper_authors.name: Yaroslav Aksenov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-13T20:36:09.037Z
paper_authors._id: 63ed5676684767daecac6f8a
paper_authors.avatarUrl: /avatars/d0e4a715f9c3fb6d74c183bab751ec35.svg
paper_authors.fullname: Yaroslav Aksenov
paper_authors.type: user
paper_authors.user: yaraksen
paper_authors._id: 67a59c4e7ffacd843a56404d
paper_authors.name: Daniil Gavrilov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-07T09:58:06.718Z
paper_authors._id: 62a9c8edc19f92ae443ab37f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669110208492-62a9c8edc19f92ae443ab37f.png
paper_authors.fullname: Daniil Gavrilov
paper_authors.type: user
paper_authors.user: kefirski
paper_title: Analyze Feature Flow to Enhance Interpretation and Steering in Language
    Models
paper_summary: We introduce a new approach to systematically map features discovered by
  sparse autoencoder across consecutive layers of large language models,
  extending earlier work that examined inter-layer feature links. By using a
  data-free cosine similarity technique, we trace how specific features persist,
  transform, or first appear at each stage. This method yields granular flow
  graphs of feature evolution, enabling fine-grained interpretability and
  mechanistic insights into model computations. Crucially, we demonstrate how
  these cross-layer feature maps facilitate direct steering of model behavior by
  amplifying or suppressing chosen features, achieving targeted thematic control
  in text generation. Together, our findings highlight the utility of a causal,
  cross-layer interpretability framework that not only clarifies how features
  develop through forward passes but also provides new means for transparent
  manipulation of large language models.
paper_ai_summary: A new data-free cosine similarity method maps and steers features across layers of large language models, enhancing interpretability and targeted control in text generation.
paper_ai_keywords: sparse autoencoder large language models data-free cosine similarity feature evolution forward passes