title: Mixture of Experts Made Intrinsically Interpretable
summary: Neurons in large language models often exhibit polysemanticity,
  simultaneously encoding multiple unrelated concepts and obscuring
  interpretability. Instead of relying on post-hoc methods, we present
  MoE-X, a Mixture-of-Experts (MoE) language model designed to be
  intrinsically interpretable. Our approach is motivated by the
  observation that, in language models, wider networks with sparse activations
  are more likely to capture interpretable factors. However, directly training
  such large sparse networks is computationally prohibitive. MoE architectures
  offer a scalable alternative by activating only a subset of experts for any
  given input, inherently aligning with interpretability objectives. In MoE-X, we
  establish this connection by rewriting the MoE layer as an equivalent sparse,
  large MLP. This approach enables efficient scaling of the hidden size while
  maintaining sparsity. To further enhance interpretability, we enforce sparse
  activation within each expert and redesign the routing mechanism to prioritize
  experts with the highest activation sparsity. These designs ensure that only
  the most salient features are routed and processed by the experts. We evaluate
  MoE-X on chess and natural language tasks, showing that it achieves performance
  comparable to dense models while significantly improving interpretability.
  MoE-X achieves a perplexity better than GPT-2, with interpretability surpassing
  even sparse autoencoder (SAE)-based approaches.
paper_authors._id: 67d0e3ede3afecf451915d0a
paper_authors.name: Xingyi Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:29:44.341Z
paper_authors._id: 634cfebc350bcee9bed20a4d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/634cfebc350bcee9bed20a4d/fN47nN5rhw-HJaFLBZWQy.png
paper_authors.fullname: Xingyi Yang
paper_authors.type: user
paper_authors.user: adamdad
paper_authors._id: 67d0e3ede3afecf451915d0b
paper_authors.name: Constantin Venhoff
paper_authors._id: 67d0e3ede3afecf451915d0c
paper_authors.name: Ashkan Khakzar
paper_authors._id: 67d0e3ede3afecf451915d0d
paper_authors.name: Christian Schroeder de Witt
paper_authors._id: 67d0e3ede3afecf451915d0e
paper_authors.name: Puneet K. Dokania
paper_authors._id: 67d0e3ede3afecf451915d0f
paper_authors.name: Adel Bibi
paper_authors._id: 67d0e3ede3afecf451915d10
paper_authors.name: Philip Torr
paper_title: Mixture of Experts Made Intrinsically Interpretable
paper_summary: Neurons in large language models often exhibit polysemanticity,
  simultaneously encoding multiple unrelated concepts and obscuring
  interpretability. Instead of relying on post-hoc methods, we present
  MoE-X, a Mixture-of-Experts (MoE) language model designed to be
  intrinsically interpretable. Our approach is motivated by the
  observation that, in language models, wider networks with sparse activations
  are more likely to capture interpretable factors. However, directly training
  such large sparse networks is computationally prohibitive. MoE architectures
  offer a scalable alternative by activating only a subset of experts for any
  given input, inherently aligning with interpretability objectives. In MoE-X, we
  establish this connection by rewriting the MoE layer as an equivalent sparse,
  large MLP. This approach enables efficient scaling of the hidden size while
  maintaining sparsity. To further enhance interpretability, we enforce sparse
  activation within each expert and redesign the routing mechanism to prioritize
  experts with the highest activation sparsity. These designs ensure that only
  the most salient features are routed and processed by the experts. We evaluate
  MoE-X on chess and natural language tasks, showing that it achieves performance
  comparable to dense models while significantly improving interpretability.
  MoE-X achieves a perplexity better than GPT-2, with interpretability surpassing
  even sparse autoencoder (SAE)-based approaches.
paper_ai_summary: MoE-X, a Mixture-of-Experts language model, achieves performance comparable to dense models while enhancing interpretability through sparse activation and prioritized expert routing.
paper_ai_keywords: Mixture-of-Experts MoE polysemanticity interpretable sparse activations large MLP expert routing perplexity