In order for machine learning to be deployed and trusted in many applications, it is crucial to be able to reliably explain why the machine learning algorithm makes certain predictions. For example, if an algorithm classifies a given pathology image to be a malignant tumor, then the doctor may need to know which parts of the image led the algorithm to this classification. How to interpret black-box predictors is thus an important and active area of research. A fundamental question is: how much can we trust the interpretation itself? In this paper, we show that interpretation of deep learning predictions is extremely fragile in the following sense: two perceptively indistinguishable inputs with the same predicted label can be assigned very different interpretations. We systematically characterize the fragility of the interpretations generated by several widely-used feature-importance interpretation methods (saliency maps, integrated gradient, and DeepLIFT) on ImageNet and CIFAR-10. Our experiments show that even small random perturbation can change the feature importance and new systematic perturbations can lead to dramatically different interpretations without changing the label. We extend these results to show that interpretations based on exemplars (e.g. influence functions) are similarly fragile. Our analysis of the geometry of the Hessian matrix gives insight on why fragility could be a fundamental challenge to the current interpretation approaches.
The paper shows that interpretations for DNN decisions, e.g. computed by methods such as sensitivity analysis or DeepLift, are fragile: Visually (to a human) inperceptibly different image cause greatly different explanations (and also to an extent different classifier outputs). The authors perturb input images and create explanations using different methods. Even though the image is inperceptibly different to a human observer, the authors observe large changes in the heatmaps visualizing the explanation maps. This is true even for random perturbations. 
The images have been modified wrt. to some noise, such that they deviate from the natural statistics for images of that kind. Since the explanation algorithms investigated in this papers merely react to the interactions of the model to the input and thus are unsupervised processes in nature, the explanation methods merely show the model's reaction to the change.
For one, the model itself reacts to the perturbation, which can be measured by the (considarbly) increased class probability. Since the prediction score is given in probabilty values, the reviewer assumes the final layer of the model is a SoftMax activation. In order to see change in the softmax output score, especially if the already dominant prediction score is further increased, a lot of change has to happen to the outputs of the layer serving as input to the SoftMax layer.
It can thus be expected, that the input- and class specific explanations change as well, to an also not so small extent. The explanation maps mirror for the considered methods the model's reaction to the input. They are thus not meaningless, but are a measure to model reaction instead of an independent process. The excellent Figure 2 supports this point. Not the interpretation itself is fragile, but the model.
Adding a small delta to the sample x shifts its position in data space, completely altering the prediction rule applied by the model due to the change in proximity to another section of the decision hyperplane. The fragility of DNN models to marginally perturbed inputs themselves is well known. 
This especially true for adversial perturbations, which have been used as test cases in this work. The explanation methods are expected to highlight highly important areas in an image, which have been targetet by these perturbation approaches.
The authors give an example of an adversary manipulating the input in order to draw the activation to specific features to draw confusing/malignant explanation maps. In a settig of model verification, the explanation via heatmaps is exactly what one wants to have: If tiny change to the image causes lots of change to the prediction (and explanation) we can visualize the instability of the model not the explanation method.
Further do targeted perturbations not show the fragility of explanation methods, but rather that the models actually find what is important to the model. It can be expected, that after a change to these parts of the input, the model will decide differently, albeit coming to the same conclusion (in terms of predicted class membership), which reflects in the explanation map computed for the perturbed input.
Further remarks:
It would be interesting to see the size and position of the center of mass attacks in the appendix. The reviewer closely follows and is experienced with various explanation methods, their application and the quality of the expected explanations. The reviewer is therefore surprised by the poor quality and lack of structure in the maps obtained from the DeepLift method. Can bugs and suboptimal configurations be ruled out during the experiments? The DeepLift explanations are almost as noisy as the ones obtained for Sensitivity Analysis (i.e. the gradient at the input point). However, recent work (e.g. Samek et al., IEEE TNNLS, 2017 or Montavon et al., Digital Signal Processing, 2017) showed that decomposition-based methods (such as DeepLift) provide less noisy explanations than Sensitivity Analysis.
Have the authors considered training the net with small random perturbations added to the samples, to compare the "vanilla" model to the more robust one, which has seen noisy samples, and compared explanations? 
Why not train (finetune) the considered models using softplus activations instead of exchanging activation nodes?
Appendix B: Heatmaps through the different stages of perturbation should be normalized using a common factor, not individually, in order to better reflect the change in the explanation
Conclusion:
The paper follows an interesting approach, but ultimately takes the wrong view point:
The authors try to attribute fragility to explaining methods, which visualize/measure the reaction of the model to the perturbed inputs. A major rework should be considered.