Decomposing and Editing Predictions by Modeling Model Computation
<|reference_start|>Decomposing and Editing Predictions by Modeling Model Computation: How does the internal computation of a machine learning model transform inputs into predictions? In this paper, we introduce a task called component modeling that aims to address this question. The goal of component modeling is to decompose an ML model's prediction in terms of its components -- simple functions (e.g., convolution filters, attention heads) that are the "building blocks" of model computation. We focus on a special case of this task, component attribution, where the goal is to estimate the counterfactual impact of individual components on a given prediction. We then present COAR, a scalable algorithm for estimating component attributions; we demonstrate its effectiveness across models, datasets, and modalities. Finally, we show that component attributions estimated with COAR directly enable model editing across five tasks, namely: fixing model errors, ``forgetting'' specific classes, boosting subpopulation robustness, localizing backdoor attacks, and improving robustness to typographic attacks. We provide code for COAR at https://github.com/MadryLab/modelcomponents .<|reference_end|>
@article{shah2024decomposing,
    title={Decomposing and Editing Predictions by Modeling Model Computation},
    author={Harshay Shah, Andrew Ilyas, Aleksander Madry},
    journal={arXiv preprint arXiv:2404.11534},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.11534},
    primaryClass={cs.LG cs.AI stat.ML}
}