Training Neural Networks for Modularity aids Interpretability
<|reference_start|>Training Neural Networks for Modularity aids Interpretability: An approach to improve network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We find pretrained models to be highly unclusterable and thus train models to be more modular using an ``enmeshment loss'' function that encourages the formation of non-interacting clusters. Using automated interpretability measures, we show that our method finds clusters that learn different, disjoint, and smaller circuits for CIFAR-10 labels. Our approach provides a promising direction for making neural networks easier to interpret.<|reference_end|>
@article{golechha2024training,
    title={Training Neural Networks for Modularity aids Interpretability},
    author={Satvik Golechha, Dylan Cope, Nandi Schoots},
    journal={arXiv preprint arXiv:2409.15747},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.15747},
    primaryClass={cs.LG cs.AI}
}