We augment adversarial training (AT) with worst case adversarial training (WCAT) which improves adversarial robustness by 11% over the current stateof-the-art result in the 2 norm on CIFAR-10. We obtain verifiable average case and worst case robustness guarantees, based on the expected and maximum values of the norm of the gradient of the loss. We interpret adversarial training as Total Variation Regularization, which is a fundamental tool in mathematical image processing, and WCAT as Lipschitz regularization.
This paper explores augmenting the training loss with an additional gradient regularization term to improve the robustness of models against adversarial examples. The authors show that this training loss can be interpreted as a form of adversarial training against optimal L2 and L_infinity adversarial perturbations. This augmented training effectively reduces the Lipschitz constant of the network, leading to improved robustness against a wide variety of attack algorithms.
While I believe the results are correct and possibly significant, the paper is poorly written (especially for a 10 page submission) and comparison with prior work on reducing the Lipschitz constant of the network is lacking. The authors also made little to no effort in writing to ensure the clarity of their paper. I would like to see a completely reworked draft before opening to the idea of recommending acceptance.
Pros:
- Theoretically intuitive method for improving the model's robustness.
- Evaluation against a wide variety of attacks.
- Empirically demonstrated improvement over traditional adversarial training.
Cons:
- Lack of comparison to prior work. The authors are aware of numerous techniques for controlling the Lipschitz constant of the network for improved robustness, but did not compare to them at all.
- Poorly written. The paper contains multiple missing figure references, has a duplicated table (Tables 1 and 3), and the method is not explained well. I am confused as to how the 2-Lip loss is minimized. Also, the paper organization seems very chaotic and incoherent, e.g., the introduction section contains many technical details that would better belong in related works or methods sections.
--------------------------------------------
Revision:
I thank the authors for incorporating my suggestions and reworking the draft, and I have updated my rating in response to the revision. While I believe the organization is much cleaner and easier to follow, there is still much room for improvement. In particular, the paper does not introduce concepts in a logical order for a non-expert to follow (e.g. Reviewer 1) and leaps into the paper's core idea too quickly. I am strongly in favor of exceeding the suggested page limit of 8 pages and using that space to address these concerns.
A more pressing concern is the evaluation of prior work. The authors added a short section (Section 5.4) comparing their method to that of (Qian and Wegman, 2018). This is certainly a reasonable comparison and the results seem promising, the evaluation lacks an important dimension -- varying the value of epsilon and observing the change in robustness. This is an important aspect for defenses against adversarial examples as certain defense may be less robust but are insensitive to the adversary's strength. Showing the robustness across different adversary strengths gives a more informative view of the authors' proposed method in comparison to others. The evaluation is also lacking in breadth, ignoring other similar defenses such as (Cisse et al., 2017) and (Gouk et al., 2018).