TOWARDS BETTER UNDERSTANDING OF GRADIENT-BASED ATTRIBUTION METHODS FOR DEEP NEURAL NETWORKS
Understanding the flow of information in Deep Neural Networks (DNNs) is a challenging problem that has gain increasing attention over the last few years. While several methods have been proposed to explain network predictions, there have been only a few attempts to compare them from a theoretical perspective. What is more, no exhaustive empirical comparison has been performed in the past. In this work, we analyze four gradient-based attribution methods and formally prove conditions of equivalence and approximation between them. By reformulating two of these methods, we construct a unified framework which enables a direct comparison, as well as an easier implementation. Finally, we propose a novel evaluation metric, called Sensitivity-n and test the gradient-based attribution methods alongside with a simple perturbation-based attribution method on several datasets in the domains of image and text classification, using various network architectures.
The paper summarizes and compares some of the current explanation techniques for deep neural networks that rely on the redistribution of relevance / contribution values from the output to the input space.
The main contributions are the introduction of a unified framework that expresses 4 common attribution techniques (Gradient * Input, Integrated Gradient, eps-LRP and DeepLIFT) in a similar way as modified gradient functions and the definition of a new evaluation measure ('sensitivity n') that generalizes the earlier defined properties of 'completeness' and 'summation to delta'.
The unified framework is very helpful since it points out equivalences between the methods and makes the implementation of eps-LRP and DeepLIFT substantially more easy on modern frameworks. However, as correctly stated by the authors some of the unification (e.g. relation between LRP and Gradient*Input) has been already mentioned in prior work.
Sensitivity-n as a measure tries to tackle the difficulty of estimating the importance of features that can be seen either separately or in combination. While the measure shows interesting trends towards a linear behaviour for simpler methods, it does not persuade me as a measure of how well the relevance attribution method mimics the decision making process and does not really point out substantial differences between the different methods. Furthermore, The authors could comment on the relation between sensitivity-n and region perturbation techniques (Samek et al., IEEE TNNLS, 2017). Sensitivtiy-n seems to be an extension of the region perturbation idea to me.
It would be interesting to see the relation between the "unified" gradient-based explanation methods and approaches (e.g. Saliency maps, alpha-beta LRP, Deep Taylor, Deconvolution Networks, Grad-CAM, Guided Backprop ...) which do not fit into the unification framework. It's good that the author mention these works, still it would be great to see more discussion on the advantages/disadvantages, because these methods may have some nice theoretically properties (see e.g. the discussion on gradient vs. decompositiion techniques in Montavon et al., Digital Signal Processing, 2017) which can not be incorporated into the unified framework.