Despite the remarkable development of recent deep learning techniques, neural networks are still vulnerable to adversarial attacks, i.e., methods that fool the neural networks with perturbations that are too small for human eyes to perceive. Many adversarial training methods were introduced as to solve this problem, using adversarial examples as a training data. However, these adversarial attack methods used in these techniques are fixed, making the model stronger only to attacks used in training, which is widely known as an overfitting problem. In this paper, we suggest a novel adversarial training approach. In addition to the classifier, our method adds another neural network that generates the most effective adversarial perturbation by finding the weakness of the classifier. This perturbation generator network is trained to produce perturbations that maximize the loss function of the classifier, and these adversarial examples train the classifier with a true label. In short, the two networks compete with each other, performing a minimax game. In this scenario, attack patterns created by the generator network are adaptively altered to the classifier, mitigating the overfitting problem mentioned above. We theoretically proved that our minimax optimization problem is equivalent to minimizing the adversarial loss after all. Beyond this, we proposed an evaluation method that could accurately compare a wide-range of adversarial training algorithms. Experiments with various datasets show that our method outperforms conventional adversarial training algorithms.
The paper proposes to construct adversarial attacks by training a neural network to produce a distortion, rather than by constructing the distortion directly via PGD. It then uses this during adversarial training to produce networks that it purports to be more robust.
Pros:
-Faster speed of adversarial training could substantially democratize ability to work on adversarial ML
-Neural net attack could overcome gradient masking in some situations
Cons:
-Some of the main arguments seem incorrect, although the paper may still be strong if these incorrect arguments are removed and the experiments are further substantiated.
Details:
I think there are some interesting ideas here, which if further substantiated could be the base of a strong paper. However, I found the overall argument in the current paper to be misleading or at least mistaken. A core claim is that the neural-net generated attack is better than PGD because PGD only generates a "single attack" while the neural network generates many different attakcs. But PGD is an adaptive attack method that depends on the current model parameters, so it changes over the course of training just like the neural net attacker. Furthermore, both models approximate the same maximization over the L2 ball (in the neural net's case, rather than maximizing over the L2 ball we penalize the L2 norm, but a similar modification can easily be made to PGD). So it is not clear why we should expect the neural net to perform better, except perhaps that it is a different type of way of approximating the same objective that PGD approximates, and perhaps this approximation has some favorable properties? But this is never justified.
Here is one attempt to justify why the neural net attack should be better: perhaps while PGD often works well, it gets stuck on certain examples, e.g. due to gradient masking issues. While the neural net might initially get stuck on those examples (it also has to work by taking gradients), since it can share knowledge across examples it initially learns a good direction for the "stuck" examples by generalizing from the "unstuck" examples. This overall allows it to generate a more consistently good attack even in the presence of (partial) gradient masking. If we believe this story, then actually the neural net attacker should already work well just as an attack (i.e. just train it to generate good attacks against a fixed model for some set of examples). I would find this quite interesting if true but unfortunately it wasn't explored in the paper. One way to do it would be to take models that are believed to be robust based on attacking them with PGD, and then show that the neural net attacker is substantially more accurate.
I do find the point about faster speed interesting. This is given as a minor comment but it may be the biggest benefit of the current method. Adversarial training on large-scale datasets like ImageNet is computationally infeasible, and your method could potentially address this and allow people who don't have $100k+ compute budgets to actually work on adversarial ML.
I also have some comments on the evaluation:
-Reporting average accuracy across attacks in the evaluation is an inappropriate summary statistics. Min accuracy would be better, which in this case is zero for all methods. This suggests that you either did not adversarially train appropriately, or used too large of a norm (what norm did you use anyways? Or is this allowing the norm to be unbounded until you fool the model? In that case all the numbers should be zero for any reasonable attack.)
-For evaluation I would rather see the full curve of accuracy vs. allowed attack norm, rather than just a single summary metric. It's too hard to tell what is going on from a single number. It is also hard to compare against adversarial training methods that penalize the L2 norm versus constrain the L2 norm, as the former might do better than the latter simply due to more closely conforming to your evaluation of average distortion. Having the full curve helps better assess this. Average distortion is also only meaningful if the method is finding the minimum-norm attack point that changes the label, which most of the methods you consider do not do. You do include the full curve in Figure 4 (please make the font bigger though). Under the full curve it seems the improvement over other methods is minimal, and could possibly be due to hyperparameter tuning.
Finally, more minor but the Gaussian derivation does not make sense. It is unlikely that your norms actually follow a Gaussian distribution. A more appropriate claim would be that you are constraining or penalizing the expected norm of the perturbations. I would remove the math part as it does not add anything to the paper (and is also wrong as per preceding comment) and focus more on how you actually construct the generator network (this is only briefly discussed in the appendix and not at all in the main text, even though it is a key point to getting the method to work). You could also use the space for more detailed experiments, following best practices as in https://arxiv.org/abs/1902.06705 to ensure that your evaluation is sensible.