PAC-learning in the presence of evasion adversaries
The existence of evasion attacks during the test phase of machine learning algorithms represents a significant challenge to both their deployment and understanding. These attacks can be carried out by adding imperceptible perturbations to inputs to generate adversarial examples and finding effective defenses and detectors has proven to be difficult. In this paper, we step away from the attack-defense arms race and seek to understand the limits of what can be learned in the presence of an evasion adversary. In particular, we extend the Probably Approximately Correct (PAC)-learning framework to account for the presence of an adversary. We first define corrupted hypothesis classes which arise from standard binary hypothesis classes in the presence of an evasion adversary and derive the Vapnik-Chervonenkis (VC)-dimension for these, denoted as the adversarial VC-dimension. We then show that sample complexity upper bounds from the Fundamental Theorem of Statistical learning can be extended to the case of evasion adversaries, where the sample complexity is controlled by the adversarial VC-dimension. We then explicitly derive the adversarial VC-dimension for halfspace classifiers in the presence of a sample-wise norm-constrained adversary of the type commonly studied for evasion attacks and show that it is the same as the standard VC-dimension. Finally, we prove that the adversarial VC-dimension can be either larger or smaller than the standard VC-dimension depending on the hypothesis class and adversary, making it an interesting object of study in its own right.
This paper develops a PAC framework for learning binary functions in the presence of evasion adversaries. Let X be the domain, and suppose we have an unknown function f : X -> {0,1} to be learned, and we also have a hypothesis class H. Moreover, there is a "closeness" relationship defined on pairs in X, so any pair of points in X are either "close" or "far." We also have an unknown distribution P on X. For a hypothesis h, its loss is defined as follows: first we pick a data point x in X according to P. The true label of this point is f(x). If there is a point y "close" to x such that h(y) differs from f(x), then we say x is classified incorrectly under the adversary (the intuition is that, the adversary can slightly perturb the point x into point y to fool the classifier). The loss of a classifier is the probability that a random point x is classified incorrectly under the adversary.  Now that we have a loss function, we can ask the standard question in learning theory: given a finite labelled sample from the unknown distribution P, can we pick a hypothesis whose loss is close to being optimal? When there is no adversary, the answer to this question is known, and the sample complexity is characterized by the VC-dimension of the hypothesis class. In this paper, a notion of "adversarial VC-dimension" has been defined, and it is proved that if this dimension is finite, then learning is possible, and a bound is given in terms of this dimension (Theorem 1), which resembles the bound on the sample complexity in terms of VC-dimension in the non-adversarial setting. The proof involves carefully reducing the problem to an appropriate adversary-free learning problem, and using a Rademacher complexity analysis.  Then the paper proceeds to understanding the connection between adversarial VC-dimension and standard VC-dimension. An example is given where the former can be arbitrarily larger than the latter. However, it is shown that if the closeness relationship is given by a norm-derived metric, then the VC-dimension of half-space classifiers do not change by adding an adversary. This is rather surprising.  I am not familiar with the literature on adversarial learning theory, so I cannot compare the framework and results with the previous ones.  Pros: + a clean framework and analysis for binary classification in the presence of adversaries is introduced, which is likely to open doors to new, interesting research. This would be of significance interest to the statistical and learning theory community.  + the framework and proofs are novel and mathematically elegant.  + the paper is well written.  Cons: - the authors have shown that finite adversarial VC-dimension implies learnability. Is the reverse also true? Does learnability under adversary imply finite adversarial VC-dimension? (Note that this is the easy direction in binary classification without adversaries, but it is not addressed in the current paper.)  Questions and comments for the authors: * Line 89: add index n for hat(h)  (compare with line 104) * 7th line in Table 1: delete "adversarial" * Line 103: "we say" learning is possible if etc. * Line 104: if there is an algorithm that, "upon receiving n training examples", etc. * Line 104: add "as n -> infinity". * Line 105: Rewrite it as: The learner does not have access to the true distribution P, but can approximate it with the empirical distribution. * Line 108: ERM -> AERM * Line 113: be "a" learning algorithm * In Lemma 1, you assume the learning algorithm does not know R. This assumption is without loss of generality, but please explain it in the paper. * Line 131: compute -> define * Display after Line 151: the dot should be replaced with "such that" or ":" * In Definition 4, explain the connection between H and F. * Line 175: C -> tilde(C) * Elaborate on Footnote 3: how can this be improved via chaining? What is the obtained result? * Line 191: "is" is missing * Line 203: what do you mean by "other variants of halfspace classifiers"? * Line 218: inside the formula, replace the dot with "we have" == after reading other reviews and authors' response: please accommodate reviewers' comments and also explain other theoretical work on adversarial learning and how your work differs from them.