Deep learning interpretation is essential to explain the reasoning behind model predictions. Understanding the robustness of interpretation methods is important especially in sensitive domains such as medical applications since interpretation results are often used in downstream tasks. Although gradient-based saliency maps are popular methods for deep learning interpretation, recent works show that they can be vulnerable to adversarial attacks. In this paper, we address this problem and provide a certifiable defense method for deep learning interpretation. We show that a sparsified version of the popular SmoothGrad method, which computes the average saliency maps over random perturbations of the input, is certifiably robust against adversarial perturbations. We obtain this result by extending recent bounds for certifiably robust smooth classifiers to the interpretation setting. Experiments on ImageNet samples validate our theory.
This paper introduces an extension of Cohen et al. (2019)’s result that allows one to derive robustness certificates for interpretation methods, as well as a bound on the top-K overlap of saliency methods. These results motivate the introduction of Sparsified SmoothGrad and a relaxation of this method that has differentiable elements. These introduced approaches adapt previous methods so the derived bounds are applicable. The proposed methods are shown to perform as well as Quadratic SmoothGrad (Smilkov et al. 2017) in CIFAR-10 experiments.
I’m not familiar with the field so it is hard for me to judge how novel the presented results are or whether the used baselines are the proper ones. That being said, the paper presents an interesting idea and it is relatively easy to read (I really appreciate the fact that for every theorem there is an interpretation, in words, for it). The only thing that sometimes makes the paper hard to read is when it starts to refer to too many constants without remind the reader what they are about. I have two complaints/questions about the relevance of the introduced bounds though. Right now, to me, it seems that the derived theoretical guarantees are not that relevant, hopefully the questions below will help clarify that.
In page 6, before introducing the “Sparsified SmoothGrad and its Relaxations”, it is said that q is set to 2^13 because otherwise the gap would be too large in images from ImageNet, for example, when comparing to traditional values of q. However, ImageNet is never revisited in the paper. I was expecting to see ImageNet results in the experimental section but they are not there (or maybe some correlation between the gap and performance -- robustness). More than that, the Quadratic SmoothGrad, which doesn’t have any theoretical guarantee, seems to perform as well as the proposed methods. So where is the gap/theoretical result relevant? What are the settings in which having a method with the derived theoretical guarantees shine? What are the limitations of Quadratic SmoothGrad? Right now, it seems to me that the “Sparsified SmoothGrad and its Relaxations” and its empirical analysis weaken the paper, because they take a big chunk of it when there is not enough evidence to claim them as an important contribution. Am I missing something? I gave this paper a relatively low score because I’m not certain about the relevance of its results, but if my questions are satisfactory answered, I’ll be happy to update my score.
------
>>> Update after rebuttal: I stand by my score after the rebuttal. 
Unfortunately I'm not an expert in this area and I don't feel confident in having a very strong opinion about this paper. That being said, enough presentation issues were raised that make me uneasy about raising my score. I do agree with some of the concerns raised by other reviewers.