How to use and interpret activation patching
<|reference_start|>How to use and interpret activation patching: Activation patching is a popular mechanistic interpretability technique, but has many subtleties regarding how it is applied and how one may interpret the results. We provide a summary of advice and best practices, based on our experience using this technique in practice. We include an overview of the different ways to apply activation patching and a discussion on how to interpret the results. We focus on what evidence patching experiments provide about circuits, and on the choice of metric and associated pitfalls.<|reference_end|>
@article{heimersheim2024how,
    title={How to use and interpret activation patching},
    author={Stefan Heimersheim, Neel Nanda},
    journal={arXiv preprint arXiv:2404.15255},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.15255},
    primaryClass={cs.LG}
}