The (Un)reliability of saliency methods
Saliency methods aim to explain the predictions of deep neural networks. These methods lack reliability when the explanation is sensitive to factors that do not contribute to the model prediction. We use a simple and common pre-processing step ---adding a mean shift to the input data--- to show that a transformation with no effect on the model can cause numerous methods to incorrectly attribute. We define input invariance as the requirement that a saliency method mirror the sensitivity of the model with respect to transformations of the input. We show, through several examples, that saliency methods that do not satisfy a input invariance property are unreliable and can lead to misleading and inaccurate attribution.
 Saliency methods are effective tools for interpreting the computation performed by DNNs, but evaluating the quality of interpretations given by saliency methods are often largely heuristic. Previous work has tried to address this shortcoming by proposing that saliency methods should satisfy "implementation invariance", which says that models that compute the same function should be assigned the same interpretation. This paper builds on this work by proposing and studying "input invariance", a specific kind of implementation invariance between two DNNs that compute identical functions but where the input is preprocessed in different ways. Then, they examine whether a number of existing saliency methods satisfy this property. The property of "implementation invariance" proposed in prior work seems poorly motivated, since the entire point of interpretations is that they should explain the computation performed by a specific network. Even if two DNNs compute the same function, they may do so using very different computations, in which case it seems natural that their interpretations should be different. Nevertheless, I can believe that the narrower property of input invariance should hold for saliency methods. A much more important concern I have is that the proposed input invariance property is not well motivated. A standard preprocessing step for DNNs is to normalize the training data, for example, by subtracting the mean and dividing by the standard deviation. Similarly, for image data, pixel values are typically normalized to [0,1]. Assuming inputs are transformed in this way, the input invariance property (for mean shift) is always trivially satisfied. The paper does not justify why we should consider networks where the training data is not normalized is such a way. Even if the input is not normalized, the failures they find in existing saliency methods are typically rather trivial. For example, for the gradient times input method, they are simply noting that the interpretation is translated by the gradient times the mean shift. The paper does not discuss why this shift matters. It is not at all clear to me that the quality of the interpretation is adversely affected by these shifts. I believe the notion that saliency methods should be invariant to input transformations may be promising, but more interesting transformations must be considered -- as far as I can tell, the property of invariance to linear transformations to the input does not provide any interesting insight into the correctness of saliency methods.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
The (Un)reliability of saliency methods
Saliency methods aim to explain the predictions of deep neural networks. These methods lack reliability when the explanation is sensitive to factors that do not contribute to the model prediction. We use a simple and common pre-processing step ---adding a mean shift to the input data--- to show that a transformation with no effect on the model can cause numerous methods to incorrectly attribute. We define input invariance as the requirement that a saliency method mirror the sensitivity of the model with respect to transformations of the input. We show, through several examples, that saliency methods that do not satisfy a input invariance property are unreliable and can lead to misleading and inaccurate attribution.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Saliency methods are effective tools for interpreting the computation performed by DNNs, but evaluating the quality of interpretations given by saliency methods are often largely heuristic. Previous work has tried to address this shortcoming by proposing that saliency methods should satisfy "implementation invariance", which says that models that compute the same function should be assigned the same interpretation. This paper builds on this work by proposing and studying "input invariance", a specific kind of implementation invariance between two DNNs that compute identical functions but where the input is preprocessed in different ways. Then, they examine whether a number of existing saliency methods satisfy this property. The property of "implementation invariance" proposed in prior work seems poorly motivated, since the entire point of interpretations is that they should explain the computation performed by a specific network. Even if two DNNs compute the same function, they may do so using very different computations, in which case it seems natural that their interpretations should be different. Nevertheless, I can believe that the narrower property of input invariance should hold for saliency methods. A much more important concern I have is that the proposed input invariance property is not well motivated. A standard preprocessing step for DNNs is to normalize the training data, for example, by subtracting the mean and dividing by the standard deviation. Similarly, for image data, pixel values are typically normalized to [0,1]. Assuming inputs are transformed in this way, the input invariance property (for mean shift) is always trivially satisfied. The paper does not justify why we should consider networks where the training data is not normalized is such a way. Even if the input is not normalized, the failures they find in existing saliency methods are typically rather trivial. For example, for the gradient times input method, they are simply noting that the interpretation is translated by the gradient times the mean shift. The paper does not discuss why this shift matters. It is not at all clear to me that the quality of the interpretation is adversely affected by these shifts. I believe the notion that saliency methods should be invariant to input transformations may be promising, but more interesting transformations must be considered -- as far as I can tell, the property of invariance to linear transformations to the input does not provide any interesting insight into the correctness of saliency methods.
assistant
 Saliency methods are effective tools for interpreting the computation performed by DNNs, but evaluating the quality of interpretations given by saliency methods are often largely heuristic. Previous work has tried to address this shortcoming by proposing that saliency methods should satisfy "implementation invariance", which says that models that compute the same function should be assigned the same interpretation. This paper builds on this work by proposing and studying "input invariance", a specific kind of implementation invariance between two DNNs that compute identical functions but where the input is preprocessed in different ways. Then, they examine whether a number of existing saliency methods satisfy this property. The property of "implementation invariance" proposed in prior work seems poorly motivated, since the entire point of interpretations is that they should explain the computation performed by a specific network. Even if two DNNs compute the same function, they may do so using very different computations, in which case it seems natural that their interpretations should be different. Nevertheless, I can believe that the narrower property of input invariance should hold for saliency methods. A much more important concern I have is that the proposed input invariance property is not well motivated. A standard preprocessing step for DNNs is to normalize the training data, for example, by subtracting the mean and dividing by the standard deviation. Similarly, for image data, pixel values are typically normalized to [0,1]. Assuming inputs are transformed in this way, the input invariance property (for mean shift) is always trivially satisfied. The paper does not justify why we should consider networks where the training data is not normalized is such a way. Even if the input is not normalized, the failures they find in existing saliency methods are typically rather trivial. For example, for the gradient times input method, they are simply noting that the interpretation is translated by the gradient times the mean shift. The paper does not discuss why this shift matters. It is not at all clear to me that the quality of the interpretation is adversely affected by these shifts. I believe the notion that saliency methods should be invariant to input transformations may be promising, but more interesting transformations must be considered -- as far as I can tell, the property of invariance to linear transformations to the input does not provide any interesting insight into the correctness of saliency methods.