Adversarially Robust Generalization Requires More Data
Machine learning models are often susceptible to adversarial perturbations of their inputs. Even small perturbations can cause state-of-the-art classifiers with high "standard" accuracy to produce an incorrect prediction with high confidence. To better understand this phenomenon, we study adversarially robust learning from the viewpoint of generalization. We show that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of "standard" learning. This gap is information theoretic and holds irrespective of the training algorithm or the model family. We complement our theoretical results with experiments on popular image classification datasets and show that a similar gap exists here as well. We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.
The paper analyses the robustness of machine learning algorithms to adversarial examples. For two particular artificial data distributions of binary classification data---overlapping Gaussians and overlapping Bernoulli distributions---the paper analyzes the sample complexity required to achieve robustness against adversarial examples. This sample complexity is compared to classical ones for the generalization error. The theoretical findings are supported by experiments on datasets that share properties with the artificial data distributions.   The quality of the paper is good, the theoretical parts are sound and well-explained in the appendix. Even though the analyzed data distributions are very limited, the results are still very insightful; in particular, because they hold for arbitrary learning algorithms. The empirical evaluation is insightful and properly related to the theoretical findings.   The paper is well-written and clear. The theoretical results are undertandable, their proofs are comprehensible. I have some minor comments to improve the clarity at the end of this review.   The main contribution of the paper are robust sample complexity bounds for two specific data distributions for binary classification for arbitrary learning algorithms (and model classes). This adds considerably to the understanding of robustness against adversarial examples. The empirical evalutation support these theoretical results. In contrast to the other theoretical findings the results on thresholding appear to be a bit trivial though: if you map every perturbation of a binary vector back to a binary vector using thresholding, then trivially the perturbations have no effect as long as they do not change the sign of elements of the binary vector.  The paper is a useful contribution to the community, because it extends the understanding of robustness against adversarial examples. Since the theoretical analysis holds for arbitrary learning algorithms, it emphasizes that adversarial examples are a general problem for learning algorithms that can be tackled by using more training examples. However, since max-margin classifiers are known to be more robust against adversarial examples, it would be great to discuss them in relation to especially deep neural networks in more detail.  The author's response has addressed my concerns about the thresholding approach, as well as my remark to further investigate max-margin classifiers.  Detailed Comments:  - The sample complexity of robust generalization is larger by sqrt(d), i.e., sub-linear in the dimension, not polynomial. - I would have liked the authors to explain more on the relation of the sample complexity of max-margin models to the robust sample complexity. - In theorem 4, I think \widehat{w} = yx should be \widehat{w} = yx / \|x\|_2, same in Theorem 5. - Line 160: the reference Corollary 23 should be Corollary 22. - In Theorem 4 and 8 it should be stated that achieving this bound with a single example is only possible because of the symmetrie of the positive and negative class distributions (at first glance, the result was a bit surprising). - I can see some benefit in shortening the formulas in Theorem 4 and Theorem 5 using constants c, c_1, c_2. However, since c_1 = 1/32 and c_2 = 64, they could also be included. Same for c= (5\sqrt{\log 1/\beta})^-1 for \beta = 0.01 (i.e., c = 5\sqrt(2)), where even the desired error bound \beta is part of the constant. - Some restrictions on possible variable assignments are not given explicitely: e.g., in Corollary 19, the error is assumed to be < 1, in Corollary 22 the \epsilon-Radius is assumed to be <= 1/4. - In the introduction, when introducing l_\infty perturbations, I suggest noting that this means robustness against adversarial examples generated within the l_\infty hypercube around a given example. - In the motivating example in the introduction: since Figure 1 shows accuracies, I suggest not using the term error---even though the two are related---since it is a bit confusing at first. Also, it might be interesting to also see the standard train accuracy.