title: Not All Language Model Features Are Linear
summary: Recent work has proposed the linear representation hypothesis: that language
  models perform computation by manipulating one-dimensional representations of
  concepts ("features") in activation space. In contrast, we explore whether some
  language model representations may be inherently multi-dimensional. We begin by
  developing a rigorous definition of irreducible multi-dimensional features
  based on whether they can be decomposed into either independent or
  non-co-occurring lower-dimensional features. Motivated by these definitions, we
  design a scalable method that uses sparse autoencoders to automatically find
  multi-dimensional features in GPT-2 and Mistral 7B. These auto-discovered
  features include strikingly interpretable examples, e.g. circular features
  representing days of the week and months of the year. We identify tasks where
  these exact circles are used to solve computational problems involving modular
  arithmetic in days of the week and months of the year. Finally, we provide
  evidence that these circular features are indeed the fundamental unit of
  computation in these tasks with intervention experiments on Mistral 7B and
  Llama 3 8B, and we find further circular representations by breaking down the
  hidden states for these tasks into interpretable components.
paper_authors._id: 66501b5bd62f7e57e34d7ac8
paper_authors.name: Joshua Engels
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-05-24T10:58:22.151Z
paper_authors._id: 6610bbffb5e6420d03bb3b05
paper_authors.avatarUrl: /avatars/8ab37ccf48410f6fc9f0f4fa851060fe.svg
paper_authors.fullname: Josh Engels
paper_authors.type: user
paper_authors.user: JoshEngels
paper_authors._id: 66501b5bd62f7e57e34d7ac9
paper_authors.name: Isaac Liao
paper_authors._id: 66501b5bd62f7e57e34d7aca
paper_authors.name: Eric J. Michaud
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-05-24T11:59:59.324Z
paper_authors._id: 6303e59c7b50dd9d0a367375
paper_authors.avatarUrl: /avatars/353aaaae6db1c7dc89895927a65fa9b1.svg
paper_authors.fullname: Eric Michaud
paper_authors.type: user
paper_authors.user: ericjm
paper_authors._id: 66501b5bd62f7e57e34d7acb
paper_authors.name: Wes Gurnee
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-05-24T05:58:45.489Z
paper_authors._id: 631f446fdaa9591e52319035
paper_authors.avatarUrl: /avatars/34d7ff8bc59aac0ed77a961596c4e4b1.svg
paper_authors.fullname: Wes Gurnee
paper_authors.type: user
paper_authors.user: wesg
paper_authors._id: 66501b5bd62f7e57e34d7acc
paper_authors.name: Max Tegmark
paper_title: Not All Language Model Features Are Linear
paper_summary: Recent work has proposed the linear representation hypothesis: that language
  models perform computation by manipulating one-dimensional representations of
  concepts ("features") in activation space. In contrast, we explore whether some
  language model representations may be inherently multi-dimensional. We begin by
  developing a rigorous definition of irreducible multi-dimensional features
  based on whether they can be decomposed into either independent or
  non-co-occurring lower-dimensional features. Motivated by these definitions, we
  design a scalable method that uses sparse autoencoders to automatically find
  multi-dimensional features in GPT-2 and Mistral 7B. These auto-discovered
  features include strikingly interpretable examples, e.g. circular features
  representing days of the week and months of the year. We identify tasks where
  these exact circles are used to solve computational problems involving modular
  arithmetic in days of the week and months of the year. Finally, we provide
  evidence that these circular features are indeed the fundamental unit of
  computation in these tasks with intervention experiments on Mistral 7B and
  Llama 3 8B, and we find further circular representations by breaking down the
  hidden states for these tasks into interpretable components.
paper_ai_keywords: linear representation hypothesis activation space irreducible multi-dimensional features independent features non-co-occurring lower-dimensional features sparse autoencoders circular features modular arithmetic strategic features hidden states interpretable components