Lower bounds on the robustness to adversarial perturbations
The input-output mappings learned by state-of-the-art neural networks are significantly discontinuous. It is possible to cause a neural network used for image recognition to misclassify its input by applying very specific, hardly perceptible perturbations to the input, called adversarial perturbations. Many hypotheses have been proposed to explain the existence of these peculiar samples as well as several methods to mitigate them, but a proven explanation remains elusive. In this work, we take steps towards a formal characterization of adversarial perturbations by deriving lower bounds on the magnitudes of perturbations necessary to change the classification of neural networks. The proposed bounds can be computed efficiently, requiring time at most linear in the number of parameters and hyperparameters of the model for any given sample. This makes them suitable for use in model selection, when one wishes to find out which of several proposed classifiers is most robust to adversarial perturbations. They may also be used as a basis for developing techniques to increase the robustness of classifiers, since they enjoy the theoretical guarantee that no adversarial perturbation could possibly be any smaller than the quantities provided by the bounds. We experimentally verify the bounds on the MNIST and CIFAR-10 data sets and find no violations. Additionally, the experimental results suggest that very small adversarial perturbations may occur with non-zero probability on natural samples.
This paper introduces lower bounds on the minimum adversarial perturbations that can be efficiently computed through layer-wise composition.
The idea and the approach is timely, and addresses one of the most pressing problems in Machine Learning. That being said, my main criticism is that the bounds are too loose: the minimum adversarials found through FGSM are several orders of magnitude larger then the estimated lower bounds. That might have two reasons: for one the lower bounds per layer might not be tight enough, or the adversarials found with FGSM are simply to large and not a good approximation for the real minimum adversarials perturbation. To test the second point, I’d encourage the authors to use better adversarial attacks like LBFGS or DeepFool (e.g. using the recently released Python package Foolbox which implements many different adversarial attacks). Also, the histograms in Figure 2&3 are difficult to compare. A histogram with the per-sample ratio between adversarial perturbation and lower bound would be more enlightening (especially once the bounds get tighter).