Published as a conference paper at ICLR 2018 THERMOMETER ENCODING: ONE HOT WAY TO RESIST ADVERSARIAL EXAMPLES
It is well known that it is possible to construct "adversarial examples" for neural networks: inputs which are misclassified by the network yet indistinguishable from true data. We propose a simple modification to standard neural network architectures, thermometer encoding, which significantly increases the robustness of the network to adversarial examples. We demonstrate this robustness with experiments on the MNIST, CIFAR-10, CIFAR-100, and SVHN datasets, and show that models with thermometer-encoded inputs consistently have higher accuracy on adversarial examples, without decreasing generalization. State-of-the-art accuracy under the strongest known white-box attack was increased from 93.20% to 94.30% on MNIST and 50.00% to 79.16% on CIFAR-10. We explore the properties of these networks, providing evidence that thermometer encodings help neural networks to find more-non-linear decision boundaries.
The authors present an in-depth study of discretizing / quantizing the input as a defense against adversarial examples. The idea is that the threshold effects of discretization make it harder to find adversarial examples that only make small alterations of the image, but also that it introduces more non-linearities, which might increase robustness. In addition, discretization has little negative impact on the performance on clean data. The authors also propose a version of single-step or multi-step attacks against models that use discretized inputs, and present extensive experiments on MNIST, CIFAR-10, CIFAR-100 and SVHN, against standard baselines and, on MNIST and CIFAR-10, against a version of quantization in which the values are represented by a small number of bits.
The merits of the paper is that the study is rather comprehensive: a large number of datasets were used, two types of discretization were tried, and the authors propose an attack mechanism better that seems reasonable considering the defense they consider. The two main claims of the paper, namely that discretization doesn't hurt performance on natural test examples and that better robustness (in the author's experimental setup) is achieved through the discretized encoding, are properly backed up by the experiments.
Yet, the applicability of the method in practice is still to be demonstrated. The threshold effects might imply that small perturbations of the input (in the l_infty sense) will not have a large effect on their discritized version, but it may also go the other way: an opponent might be able to greatly change the discretized input without drastically changing the input. Figure 8 in the appendix is a bit worrysome on that point, as the performance of the discretized version drops rapidly to 0 when the opponents gets a bit stronger. Did the authors observe the same kind of bahavior on other datasets? What would the authors propose to mitigate this issue? To what extend the good results that are exhibited in the paper are valid over the wide range of opponent's strengths?
minor comment:
- the experiments on CIFAR-100 in Appendix E are carried out by mixing adversarial / clean examples while training, whereas those on SVHN in Appendix F use adversarial examples only.