We show that simple spatial transformations, namely translations and rotations alone, suffice to fool neural networks on a significant fraction of their inputs in multiple image classification tasks. Our results are in sharp contrast to previous work in adversarial robustness that relied on more complicated optimization approaches unlikely to appear outside a truly adversarial context. Moreover, the misclassifying rotations and translations are easy to find and require only a few black-box queries to the target model. Overall, our findings emphasize the need to design robust classifiers even for natural input transformations in benign settings.
Summary
The authors study robustness of neural networks for image recognition tasks with respect to geometric transformations in input space. The question is posed in an adversarial setting, where the authors exploit that Conv/ResNets are not fully translation and rotation invariant. The authors propose three untargeted attacks to increase the classification error of the network: a first-order method, an attack involving random transformations and a grid search of allowed transformations. For the random and grid search the worst prediction is considered the outcome of the attack. The authors observe that first-order attacks are not very successful in fooling the network compared to the grid search. Data augmentation as a counter measure is found to be not sufficient and adversarial (robust) training with respect to the random search attack is proposed in addition.
Evaluation
The paper is well written and particularly the empirical part is interesting. However, novelty is limited, the best approach boils down to a grid search that tests multiple hypotheses instead of a single one. There are some conceptual problems and important aspects like confidences of the classification are not addressed.
Novelty:
Many claims and observations appear trivial and well-known. E.g.,
- the research question has already been addressed by related work, leaving the proposed attacks trivial given that the attack space (allowed transformations) is specified ad hoc and without a proper measure.
- that data augmentation and training with the adversarial loss function (i.e. with the attack scheme in mind) is helpful is straight forward and not surprising
Detailed comments:
The authors study whether neural networks are robust to transformations in input space and resort to a benign adversarial setting. I'm wondering whether this allows for an answer regarding general robustness? That is, the experiments are conducted wrt the worst case, while the training does not account for an attack setting. E.g, it is unclear why the classifier would not involve a pre-processing step to counter transformations in input space, see Rowley et al. (1998).
Translation and rotation invariance of neural networks has been addressed by many authors, e.g., see Jaderberg et al. (2015) and Marcos et al. (2017).
Adversarial examples are defined to be similar and misclassified with high confidence.
The similarity of the transformation is not addressed properly. E.g., if the goal of the adversary is to force errors, why not allow for rotations of 180 degrees? Pixel-based attacks (Goodfellow et al., 2014) are more rigorous in this regard while the cited transformation-based attacks (Kanbak et al. 2017; Xiao et al., 2018) are virtually indistinguishable from the real test cases.
The effectiveness of the grid search attack seems to be connected to performing $5 * 5 * 31 = 775$ individual tests for each test case where only the worst outcome would count. The sheer number should render a misclassification more likely compared to the competitors. This is supported by empirical findings showing that only a small subset of transformations per test case accounts for the misclassification on CIFAR10 and ImageNet (Fig. 10 in the Appendix).
Regarding the padding experiments, I wonder whether the network architecture is appropriate for the new input. Here, more experimentation is necessary. The conclusion with respect to the first-order method remains a conjecture.
References:
- Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
- Max Jaderberg, Karen Simonyan, and Andrew Zisserman. Spatial transformer networks. In Advances in neural information processing systems, pp. 2017–2025, 2015.
- Can Kanbak, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Geometric robustness of deep networks: analysis and improvement. arXiv preprint arXiv:1711.09115, 2017.
- Diego Marcos, Michele Volpi, Nikos Komodakis, and Devis Tuia. Rotation equivariant vector field networks. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
- Henry Rowley, Shumeet Baluja, and Takeo Kanade. Rotation invariant neural network-based face detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 38. sn, 1998.
- Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. arXiv preprint arXiv:1801.02612, 2018.