Adversarial perturbations cause a shift in the salient features of an image, which often results in misclassification. Previous work has suggested that these salient features could be used as a defense, arguing that with saliency tools we could successfully detect adversarial examples. While the idea itself is appealing, we show that prior work which used gradient-based saliency tools is ineffective as an adversarial defense -it fails to beat a simple baseline which uses the same model but with the saliency map removed. To remedy this, we demonstrate that learnt saliency models can capture the shifts in saliency due to adversarial perturbations, while also having a low computational cost. This allows saliency models to be used effectively as a real-time defense. Further, using the learnt saliency model, we propose a novel defense: a CNN that distinguishes between adversarial images and natural images using salient pixels as its input. On MNIST, CIFAR-10, and ASSIRA, our defense improves on using the saliency map alone, and can detect various adversarial attacks. Lastly, we show that even when trained on weak defenses, we can detect adversarial images generated by strong attacks such as C&W and DeepFool.
The paper studies methods for detecting adversarial examples using saliency maps. The authors propose using the method of Dabkowski and Gal (2017) to generate saliency maps and then train a classifier on these maps (or their dot product with the input image) to distinguish natural from adversarial examples. They perform experiments evaluating the white-box and black-box robustness of their detection scheme.
From a technical perspective, the contribution of the paper is rather incremental. The detection of adversarial examples by training a classifier on saliency maps has already been studied in prior work. The only modification proposed in this work is using an (existing) alternative method for producing the saliency maps and utilizing the dot product of maps with images.
From a conceptual perspective, the impact of detecting specific adversarial attacks is not clear. In a realistic setting, an adversary could use a very different attack or even utilize a different set of transformations (e.g. image rotations). Thus, in order to demonstrate the utility of their method in a black-box scenario, the authors would need to evaluate the defense in a variety of different scenarios. At the very least, they should consider generalization to difference attacks (e.g., train against FGSM and BIM, and test against DF).
Moreover, the robustness against white-box adversaries is not sufficiently studied. Firstly, the robustness of the non-adversarially trained detector is suspiciously high. There is little reason to expect that a composition of two neural networks (the saliency map methods and the classifier) would be non-trivially robust. The authors should consider alternative attacks perhaps using more iterations with a smaller step size. Secondly, after adversarial training, only the robustness against the same attack is considered. In order to argue about white-box robustness, the authors would need to evaluate against a variety of diverse adversaries.
Overall, the technical and conceptual contribution of this paper is insufficient for publication at ICLR, even ignoring the concerns about its experimental evaluation.