Published as a conference paper at ICLR 2018 ROBUSTNESS OF CLASSIFIERS TO UNIVERSAL PERTUR- BATIONS: A GEOMETRIC PERSPECTIVE
Deep networks have recently been shown to be vulnerable to universal perturbations: there exist very small image-agnostic perturbations that cause most natural images to be misclassified by such classifiers. In this paper, we provide a quantitative analysis of the robustness of classifiers to universal perturbations, and draw a formal link between the robustness to universal perturbations, and the geometry of the decision boundary. Specifically, we establish theoretical bounds on the robustness of classifiers under two decision boundary models (flat and curved models). We show in particular that the robustness of deep networks to universal perturbations is driven by a key property of their curvature: there exist shared directions along which the decision boundary of deep networks is systematically positively curved. Under such conditions, we prove the existence of small universal perturbations. Our analysis further provides a novel geometric method for computing universal perturbations, in addition to explaining their properties.
The paper develops models which attempt to explain the existence of universal perturbations which fool neural networks — i.e., the existence of a single perturbation which causes a network to misclassify most inputs. The paper develops two models for the decision boundary:
(a) A locally flat model in which the decision boundary is modeled with a hyperplane and the normals two the hyperplanes are assumed to lie near a low-dimensional linear subspace.
(b) A locally positively curved model, in which there is a positively curved outer bound for the collection of points which are assigned a given label. 
The paper works out a probabilistic analysis arguing that when either of these conditions obtains, there exists a fooling perturbation which affects most of the data. 
The theoretical analysis in the paper is straightforward, in some sense following from the definition. The contribution of the paper is to posit these two conditions which can predict the existence of universal fooling perturbations, argue experimentally that they occur in (some) neural networks of practical interest. 
One challenge in assessing the experimental claims is that practical neural networks are nonsmooth; the quadratic model developed from the hessian is only valid very locally. This can be seen in some of the illustrative examples in Figure 5: there *is* a coarse-scale positive curvature, but this would not necessarily come through in a quadratic model fit using the hessian. The best experimental evidence for the authors’ perspective seems to be the fact that random perturbations from S_c misclassify more points than random perturbations constructed with the previous method. 
I find the topic of universal perturbations interesting, because it potentially tells us something structural (class-independent) about the decision boundaries constructed by artificial neural networks. To my knowledge, the explanation of universal perturbations in terms of positive curvature is novel. The paper would be much stronger if it provided an explanation of *why* there exists this common subspace of universal fooling perturbations, or even what it means geometrically that positive curvature obtains at every data point. 
Visually, these perturbations seem to have strong, oriented local high-frequency content — perhaps they cause very large responses in specific filters in the lower layers of a network, and conventional architectures are not robust to this? 
It would also be nice to see some visual representations of images perturbed with the new perturbations, to confirm that they remain visually similar to the original images.