Adaptive Generation of Unrestricted Adversarial Inputs
Neural networks are vulnerable to adversarially-constructed perturbations of their inputs. Most research so far has considered perturbations of a fixed magnitude under some l p norm. Although studying these attacks is valuable, there has been increasing interest in the construction of-and robustness to-unrestricted attacks, which are not constrained to a small and rather artificial subset of all possible adversarial inputs. We introduce a novel algorithm for generating such unrestricted adversarial inputs which, unlike prior work, is adaptive: it is able to tune its attacks to the classifier being targeted. It also offers a 400-2,000× speedup over the existing state of the art. We demonstrate our approach by generating unrestricted adversarial inputs that fool classifiers robust to perturbation-based attacks. We also show that, by virtue of being adaptive and unrestricted, our attack is able to defeat adversarial training against it.
The paper proposes using GANs to generate unrestricted adversarial examples. They seek to generate examples that are adversarial for a specific classifier, and they do so by using class-conditional GANs and a fine-tuning loss. The fine-tuning loss consists of both the ordinary GAN loss (to fool the discriminator) as well as an adversarial loss (which rewards the GAN for generating examples misclassified by the specific classifier). The authors perform various experiments on their generated examples to check for realism and how adversarial the generated images are.
I would reject this paper for two key reasons. First, I feel that the contributions are not significant enough (in comparison to the prior work of Song et. al). Second, I feel that some of the methods (and some of the writing) are not too principled.
In my opinion, unrestricted adversarial examples are significant if they can be made to be realistic. If our current deep learning models often mislabeled very realistic images, that would properly expose a big failure mode of our current models. However, if our machine learning models perform poorly on images that look fake/generated 40% of the time (which is what the authors state) and don’t look too realistic to humans, it is less worrying.
In comparison to Song et. al, the authors state that their methods result in very similar results in terms of realism and how adversarial their images are (arguably, Song et. al actually produces better results in terms of being adversarial). In my opinion, the authors’ claimed improvements are not significant enough, because I think realism should be the primary metric to evaluate this field. Improving speed of generation is nice, and being able to bypass a simple adversarial training procedure is interesting but not significant unless this insight is expanded upon. The results on MNIST in Fig. 5 and Fig. 6 are not too convincing, as simpler attacks that generate (arguably) more realistic images like translations and rotations [1] or L1/L2 attacks [2] (since the networks are trained for L_inf robustness) can also degrade accuracy. Finally, I can also think of another reasonable baseline that I would have liked to see the authors compare their method against. Because the authors want to attack a specific network, they could have (1) generated realistic images using a pre-trained GAN (2) used a norm-bounded attack on the specific classifier and the generated GAN images. These images could be even more realistic if the norm-bound of the attack is fairly small, and would still be able to attack specific classifiers.
Finally, I am confused by the comparison to a not-fine-tuned GAN in Fig. 14/Fig. 15 and would appreciate a clarification so that I can understand the results. For example, what does it mean for intended true label = 9, target label = 0 to have 90% success in Fig. 15? Does this mean that when you try to generate a 9 with the GAN, the classifier misclassifies it as a 0 90% of the time? In particular, I’m struggling to understand what the target label is for the case of the not-fine-tuned GAN.
Secondly, I feel that there are many instances in the paper where the methods used are not explained in a principled way. For example, one of the key parts of this work is the fine-tuning loss function. Why does the loss function involve multiplying the ordinary GAN loss (with some additional transformation applied to it which seems unnecessary) with the adversarial loss? It seems most reasonable add the adversarial loss and the ordinary GAN loss (without the additional transformation). Is the stochastic loss selection procedure necessary? If all these peculiarities of the method are necessary, it seems that the success of this method is quite brittle.
Additional feedback:
- In the intro, I think citing [3] in addition to Xu et. al is more appropriate.
- You should refer to Figure 1 somewhere in the text of your work
- In section 3.2, you can use “cosine similarity” to describe what you are doing faster.
- When you talk about “global optima of realistic adversarial examples” and “local optimal of unrealistic adversarial examples,” it sounds weird. I would try to reword this because I don’t think you are trying to make a precise mathematical statement but it sounds like one when you write it this way.
- In Table 1, I would format the numbers better to be vertically aligned
- You should provide a citation for MixTrain on page 5
[1] https://arxiv.org/abs/1712.02779,
[2] https://arxiv.org/abs/1905.01034
[3] https://arxiv.org/abs/1802.00420