Efficient and Accurate Explanation Estimation with Distribution Compression
<|reference_start|>Efficient and Accurate Explanation Estimation with Distribution Compression: Exact computation of various machine learning explanations requires numerous model evaluations and in extreme cases becomes impractical. The computational cost of approximation increases with an ever-increasing size of data and model parameters. Many heuristics have been proposed to approximate post-hoc explanations efficiently. This paper shows that the standard i.i.d. sampling used in a broad spectrum of algorithms for explanation estimation leads to an approximation error worthy of improvement. To this end, we introduce Compress Then Explain (CTE), a new paradigm for more efficient and accurate explanation estimation. CTE uses distribution compression through kernel thinning to obtain a data sample that best approximates the marginal distribution. We show that CTE improves the estimation of removal-based local and global explanations with negligible computational overhead. It often achieves an on-par explanation approximation error using 2-3x less samples, i.e. requiring 2-3x less model evaluations. CTE is a simple, yet powerful, plug-in for any explanation method that now relies on i.i.d. sampling.<|reference_end|>
@article{baniecki2024efficient,
    title={Efficient and Accurate Explanation Estimation with Distribution
  Compression},
    author={Hubert Baniecki, Giuseppe Casalicchio, Bernd Bischl, Przemyslaw Biecek},
    journal={arXiv preprint arXiv:2406.18334},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.18334},
    primaryClass={cs.LG stat.ML}
}