Mixture of Experts Made Intrinsically Interpretable
Neurons in large language models often exhibit polysemanticity,
simultaneously encoding multiple unrelated concepts and obscuring
interpretability. Instead of relying on post-hoc methods, we present
MoE-X, a Mixture-of-Experts (MoE) language model designed to be
intrinsically interpretable. Our approach is motivated by the
observation that, in language models, wider networks with sparse activations
are more likely to capture interpretable factors. However, directly training
such large sparse networks is computationally prohibitive. MoE architectures
offer a scalable alternative by activating only a subset of experts for any
given input, inherently aligning with interpretability objectives. In MoE-X, we
establish this connection by rewriting the MoE layer as an equivalent sparse,
large MLP. This approach enables efficient scaling of the hidden size while
maintaining sparsity. To further enhance interpretability, we enforce sparse
activation within each expert and redesign the routing mechanism to prioritize
experts with the highest activation sparsity. These designs ensure that only
the most salient features are routed and processed by the experts. We evaluate
MoE-X on chess and natural language tasks, showing that it achieves performance
comparable to dense models while significantly improving interpretability.
MoE-X achieves a perplexity better than GPT-2, with interpretability surpassing
even sparse autoencoder (SAE)-based approaches.
67d0e3ede3afecf451915d0a
Xingyi Yang
claimed_verified
2025-06-05T07:29:44.341Z
634cfebc350bcee9bed20a4d
https://cdn-avatars.huggingface.co/v1/production/uploads/634cfebc350bcee9bed20a4d/fN47nN5rhw-HJaFLBZWQy.png
Xingyi Yang
user
adamdad
67d0e3ede3afecf451915d0b
Constantin Venhoff
67d0e3ede3afecf451915d0c
Ashkan Khakzar
67d0e3ede3afecf451915d0d
Christian Schroeder de Witt
67d0e3ede3afecf451915d0e
Puneet K. Dokania
67d0e3ede3afecf451915d0f
Adel Bibi
67d0e3ede3afecf451915d10
Philip Torr
Mixture of Experts Made Intrinsically Interpretable
Neurons in large language models often exhibit polysemanticity,
simultaneously encoding multiple unrelated concepts and obscuring
interpretability. Instead of relying on post-hoc methods, we present
MoE-X, a Mixture-of-Experts (MoE) language model designed to be
intrinsically interpretable. Our approach is motivated by the
observation that, in language models, wider networks with sparse activations
are more likely to capture interpretable factors. However, directly training
such large sparse networks is computationally prohibitive. MoE architectures
offer a scalable alternative by activating only a subset of experts for any
given input, inherently aligning with interpretability objectives. In MoE-X, we
establish this connection by rewriting the MoE layer as an equivalent sparse,
large MLP. This approach enables efficient scaling of the hidden size while
maintaining sparsity. To further enhance interpretability, we enforce sparse
activation within each expert and redesign the routing mechanism to prioritize
experts with the highest activation sparsity. These designs ensure that only
the most salient features are routed and processed by the experts. We evaluate
MoE-X on chess and natural language tasks, showing that it achieves performance
comparable to dense models while significantly improving interpretability.
MoE-X achieves a perplexity better than GPT-2, with interpretability surpassing
even sparse autoencoder (SAE)-based approaches.
MoE-X, a Mixture-of-Experts language model, achieves performance comparable to dense models while enhancing interpretability through sparse activation and prioritized expert routing.
Mixture-of-Experts MoE polysemanticity interpretable sparse activations large MLP expert routing perplexity