Constructing Unrestricted Adversarial Examples with Generative Models
Adversarial examples are typically constructed by perturbing an existing data point within a small matrix norm, and current defense methods are focused on guarding against this type of attack. In this paper, we propose unrestricted adversarial examples, a new threat model where the attackers are not restricted to small normbounded perturbations. Different from perturbation-based attacks, we propose to synthesize unrestricted adversarial examples entirely from scratch using conditional generative models. Specifically, we first train an Auxiliary Classifier Generative Adversarial Network (AC-GAN) to model the class-conditional distribution over data samples. Then, conditioned on a desired class, we search over the AC-GAN latent space to find images that are likely under the generative model and are misclassified by a target classifier. We demonstrate through human evaluation that unrestricted adversarial examples generated this way are legitimate and belong to the desired class. Our empirical results on the MNIST, SVHN, and CelebA datasets show that unrestricted adversarial examples can bypass strong adversarial training and certified defense methods designed for traditional adversarial attacks.
****** EDIT: I am upgrading my review score in response to the authors' rebuttal. However, the similarities with Song et al should be made much more explicit, and the reviewers' other comments should also be taken into account. *******  This paper proposes a technique for generating adversarial examples that are not slight perturbations of an existing data sample. Rather, they are samples that are designed to look like a certain class to an observer, but get classified to a pre-specified, different class. The approach for doing so is to define a loss function that combines these two objectives (while also enforcing that the latent code vector should be near a randomly-selected initial vector). The authors show that this method is able to generate successful adversarial examples against schemes that are designed to protect against perturbation-based defenses.   Overall, the paper seems to capture a more reasonable definition of adversarial examples; there is no inherent reason why adversarial examples should necessarily be based on an existing sample, and using GANs is an interesting way to decouple the adversarial example generation from the samples available in a dataset. Besides this insight, I didn’t feel like the ideas are particularly novel from a technical standpoint—the machine learning aspects of the paper are fairly standard. I think this work may be more interesting from a security standpoint than from a machine learning one. However, the authors did a fairly good job of evaluating their ideas (especially considering how subjective the problem is), and the paper was overall an enjoyable read.   The paper states that using a larger epsilon is not effective because the samples are obviously tampered with. At least visually, I thought the samples in Figure 1 also looked tampered with (or at least clearly distorted). The experiment seems a little strange to me, because the samples with larger epsilon mainly appear to have a different background in Figure 2. This is a very easy thing to detect, but it doesn’t mean that the image looks substantially different from a hand-written digit. In fact, to me, the images just look like MNIST digits with a fuzzed background. Perhaps this experiment would have been more informative on a different dataset, like CIFAR? My concern is that because MNIST is so simple, it is easy to see these distortions in background, whereas they might not be as visible on a richer background.   The paper is very well-written, though there are several typos that should be proofread.