title: Monet: Mixture of Monosemantic Experts for Transformers
summary: Understanding the internal computations of large language models (LLMs) is
  crucial for aligning them with human values and preventing undesirable
  behaviors like toxic content generation. However, mechanistic interpretability
  is hindered by polysemanticity -- where individual neurons respond to multiple,
  unrelated concepts. While Sparse Autoencoders (SAEs) have attempted to
  disentangle these features through sparse dictionary learning, they have
  compromised LLM performance due to reliance on post-hoc reconstruction loss. To
  address this issue, we introduce Mixture of Monosemantic Experts for
  Transformers (Monet) architecture, which incorporates sparse dictionary
  learning directly into end-to-end Mixture-of-Experts pretraining. Our novel
  expert decomposition method enables scaling the expert count to 262,144 per
  layer while total parameters scale proportionally to the square root of the
  number of experts. Our analyses demonstrate mutual exclusivity of knowledge
  across experts and showcase the parametric knowledge encapsulated within
  individual experts. Moreover, Monet allows knowledge manipulation over domains,
  languages, and toxicity mitigation without degrading general performance. Our
  pursuit of transparent LLMs highlights the potential of scaling expert counts
  to enhance} mechanistic interpretability and directly resect the internal
  knowledge to fundamentally adjust} model behavior. The source code and
  pretrained checkpoints are available at https://github.com/dmis-lab/Monet.
paper_authors._id: 675282e85b1cc8f5a17f70d2
paper_authors.name: Jungwoo Park
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T09:08:53.485Z
paper_authors._id: 60f8435644e75317cc02ed51
paper_authors.avatarUrl: /avatars/68b7fc077fe2bda6607b1c470add8140.svg
paper_authors.fullname: Jungwoo Park
paper_authors.type: user
paper_authors.user: affjljoo3581
paper_authors._id: 675282e85b1cc8f5a17f70d3
paper_authors.name: Young Jin Ahn
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-09T11:07:18.378Z
paper_authors._id: 613f076fd86a0afab8db7117
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1637046229907-613f076fd86a0afab8db7117.jpeg
paper_authors.fullname: Ahn Young Jin
paper_authors.type: user
paper_authors.user: snoop2head
paper_authors._id: 675282e85b1cc8f5a17f70d4
paper_authors.name: Kee-Eung Kim
paper_authors._id: 675282e85b1cc8f5a17f70d5
paper_authors.name: Jaewoo Kang
paper_title: Monet: Mixture of Monosemantic Experts for Transformers
paper_summary: Understanding the internal computations of large language models (LLMs) is
  crucial for aligning them with human values and preventing undesirable
  behaviors like toxic content generation. However, mechanistic interpretability
  is hindered by polysemanticity -- where individual neurons respond to multiple,
  unrelated concepts. While Sparse Autoencoders (SAEs) have attempted to
  disentangle these features through sparse dictionary learning, they have
  compromised LLM performance due to reliance on post-hoc reconstruction loss. To
  address this issue, we introduce Mixture of Monosemantic Experts for
  Transformers (Monet) architecture, which incorporates sparse dictionary
  learning directly into end-to-end Mixture-of-Experts pretraining. Our novel
  expert decomposition method enables scaling the expert count to 262,144 per
  layer while total parameters scale proportionally to the square root of the
  number of experts. Our analyses demonstrate mutual exclusivity of knowledge
  across experts and showcase the parametric knowledge encapsulated within
  individual experts. Moreover, Monet allows knowledge manipulation over domains,
  languages, and toxicity mitigation without degrading general performance. Our
  pursuit of transparent LLMs highlights the potential of scaling expert counts
  to enhance} mechanistic interpretability and directly resect the internal
  knowledge to fundamentally adjust} model behavior. The source code and
  pretrained checkpoints are available at https://github.com/dmis-lab/Monet.
paper_ai_summary: The Mixture of Monosemantic Experts for Transformers (Monet) architecture enhances mechanistic interpretability of large language models by incorporating sparse dictionary learning directly into end-to-end pretraining, allowing for increased expert count and improved knowledge manipulation.
paper_ai_keywords: Sparse Autoencoders Mixture-of-Experts pretraining mutual exclusivity of knowledge parametric knowledge toxicity mitigation mechanistic interpretability knowledge manipulation