In recent years, deep neural networks have demonstrated outstanding performance in many machine learning tasks. However, researchers have discovered that these state-of-the-art models are vulnerable to adversarial examples: legitimate examples added by small perturbations which are unnoticeable to human eyes. Adversarial training, which augments the training data with adversarial examples during the training process, is a well known defense to improve the robustness of the model against adversarial attacks. However, this robustness is only effective to the same attack method used for adversarial training. Madry et al. (2017) suggest that effectiveness of iterative multi-step adversarial attacks and particularly that projected gradient descent (PGD) may be considered the universal first order adversary and applying the adversarial training with PGD implies resistance against many other first order attacks. However, the computational cost of the adversarial training with PGD and other multi-step adversarial examples is much higher than that of the adversarial training with other simpler attack techniques. In this paper, we show how strong adversarial examples can be generated only at a cost similar to that of two runs of the fast gradient sign method (FGSM), allowing defense against adversarial attacks with a robustness level comparable to that of the adversarial training with multi-step adversarial examples. We empirically demonstrate the effectiveness of the proposed two-step defense approach against different attack methods and its improvements over existing defense strategies.
Summary. The authors propose a novel adversarial training method, e2SAD, that relies on a two-step process for generating sets of two training adversarial samples for each clean training sample. The first step is a classical FGSM that yields the first adversarial sample. The second adversarial sample is calculated with a FGSM that is based on the cross-entropy between the probabilities generated by the first adversarial sample and the probabilities generated by the second adversarial sample. The method is computationally efficient (two forward/backward passes per clean sample) w.r.t. powerful iterative attacks such as IFGSM or PGD requiring 40+ steps and the authors claim it gives comparable results to adversarial training with multi-step attacks methods in white and black-box settings.
Clarity. Part 1 and 2 of the paper are well written and summarize the existing attacks/defense mechanisms, their pros and cons as well as the contributions clearly. The next sections could be made shorter (see comments below) to match ICLR’s recommended soft limit of 8 pages instead of the 10 pages hard limit. This would also help the reader grasp the key ideas faster and have a standard formatting (no negative spaces for instance).
Novelty. The idea of simulating the effect of iterative attacks using two distinct steps is novel and appealing to me. The first step increases the loss while the second step shifts the probability distributions apart.
Pros and cons.
(+) The paper is clear and easy to follow, although a bit long.
(+) The idea is interesting and clearly motivated in terms of computational efficiency and in terms of desired properties (Figure 2 illustrates this point well).
(-) Only one aspect of the idea is exploited in the article. It would be interesting to compare this method as an attacker (both in terms of performance and in terms of generated samples, see comment below). Powerful adversarial training should indeed rely on powerful generated adversarial samples.
(-) The results seem somewhat mitigated in terms of significance and conclusions drawn by the authors. Also, the experimental setup is quite light, notably the used CNN architectures are quite small and other datasets could have been used (also linked to the significance of the results).
Comments.
- Shorter paper. Here are suggested modifications for the paper that could help strengthen the impact of your paper. Section 3.1 could be almost entirely discarded as it brings no new ideas w.r.t sections 1 and 2. Figure 1 summarizes the method well, thus the description in Section 3.2 could be made shorter, especially when displaying Equation (8) right after Figure 1. This would then help reduce the size of Sections 3.2.1 and 3.2.2 (because Equation (8) and Figure 1 would prevent you from repeating claims made earlier in the paper). Algorithm 1 is straightforward and could be placed in Appendix. Conclusion and Result sections could be shortened a little as well (not as much as Section 3 though).
- Significance of the results. The significance of some results is unclear to me. Could the authors provide the standard deviation over 3 or 5 runs? For example, in rows 1, 3, 4, 5, 6 of Table 2, it is not clear it e2SAD performs better than FGSM adversarial training, thus raising the question of the necessity of Step 2 of the attack (which is the core contribution of the paper).
- Experimental setup. The last two rows of Table 1 are encouraging for e2SAD. However, the authors could introduce another dataset, e.g. CIFAR10 or 100 or even ImageNet restricted to 20 or 100 random classes/with fewer samples per class and use deeper modern CNN architectures like ResNets (even a ResNet18). Those models are widely adopted both in the research community and by the industry, thus defense mechanisms that provably work for such models can have a huge impact.
- Defense setup. Is the order of Steps 1 and 2 relevant? What if the authors use only iterations of Step 2?
- Attack setup. Here are a few suggestions for assessing your method in an attack setting: what is the precision of the network, without any defense, given an average dissimilarity L2 budget in the training/test samples, in a white/black box setting? How does it compare to standard techniques (e.g. FGSM, IFGSM, DeepFool, Carlini)? What happens if the authors use their method both for both defense and attack? Could the authors display adversarial samples generated by their method?
Conclusion. The idea presented in the paper is interesting, but (1) the experimental results are not entirely satisfactory for the moment and (2) only one aspect of the idea is exploited in the paper, which can be made more interesting and impactful while studying both attack and defense setups. I strongly encourage the authors to continue their research in this area due to the high potential impact and benefits for the whole community.