We propose a novel data-dependent structured gradient regularizer to increase the robustness of neural networks vis-a-vis adversarial perturbations. Our regularizer can be derived as a controlled approximation from first principles, leveraging the fundamental link between training with noise and regularization. It adds very little computational overhead during learning and is simple to implement generically in standard deep learning frameworks. Our experiments provide strong evidence that structured gradient regularization can act as an effective first line of defense against attacks based on long-range correlated signal corruptions.
Short paper summary: This work proposes a novel method of gradient regularization (SGR) which utilizes the covariance structure of adversarial examples generated during training. The authors propose simple techniques to reduce the computational overhead of SGR. Empirically, the authors compare their method to standard adversarial training and gradient norm regularization.
Brief review summary: There are some interesting ideas in this work but I feel that the some practical aspects lack formal justification and the comparison to existing work is inconclusive.
Detailed comments:
In addition to some minor comments, I have two concerns. First, with the SGR algorithm itself. And second with the empirical analysis. While I suspect that the first concern may be clarified with discussion I think that the second is more serious and is the primary factor behind my review score.
1) As the SGR algorithm is written I wonder whether the regularization term may be computed more efficiently using something like a Hutchinson trace estimation trick. I suspect that if the random vector used to estimate the trace was the xi from Algorithm 1 then the same Mahalanobis gradient norm would be recovered. This would hold only in the case beta=1, bringing me to my second point.
2) What is the purpose of the running average of the covariance? A relatively small beta value is used in practice but I do not see any strong justification for this. Is there a good reason why we do not want the covariance matrix to be a close approximation for the local gradient landscape? This seems like an important part of the algorithm, especially as it may shed light on my next note.
3) In practice, Algorithm 1 uses adversarial attack schemes to generate the perturbations. In simple cases like FGM, this would give the covariance of the input-output gradient which seems that it would have a direct interpretation as a form of classical gradient regularization. To this extent, I also wonder how the SGR algorithm could be related to interpretations of adversarial training as gradient smoothing (when using small perturbations).
I recognize that the above points are (so far as I could tell) not directly addressed in the work, and some may be fairly considered out of scope. However, due to the direct comparison to adversarial training later and the need to tie SGR to adversarial attacks I feel that it would be important to distinguish these cases.
Overall, I felt that the first three sections did well to introduce the motivation and techniques used and were was easy to follow. The derivation of the SGR algorithm was clear and concise but I believe that some of the practical details (covariance running average, computational efficiency [at first glance, it looks like the full Jacobian must be computed, but practically the sum over K reduces this to a single backprop call]) could have been elaborated on.
For the empirical evaluation the authors provided ample detail on the experimental set up and have performed a fairly thorough investigation in terms of existing defenses and attacks. I felt that the bulk of the study which is contained in Table 1 is fairly inconclusive or at the very least, difficult to interpret completely. Additional comments:
4) I felt that Figure 1 and 2 are a little difficult to interpret at first. It would help to clearly define what is meant by short- and long-range signal corruptions. However, they do suggest some interesting findings. As these covariance matrices depend directly on the model itself, I think it is worth investigate (or commenting on) how this structure may change when introducing things like SGR (or GN). The authors claim that unregularized classifiers give too much weight to short range correlations but they should show that SGN (or other methods) correct this.
5) My biggest concern with this work is with the results presented in Table 1. In terms of how they are presented: first I think that the fool column requires further explanation, or perhaps more simply the column could show accuracy instead of the average perturbation size. Second, I am not sure why the reported accuracies are averaged over attack strengths in a range. So far as I am aware, this is not standard and makes it difficult to interpret the performance of the models in this way. Figure 4 in the appendix does a better job of describing the behavior over a range of attack strengths.
6) From the table, it is not obvious to me that SGR provides any improvements to robustness over existing techniques. Indeed, the authors write that SGR achieves white-box accuracies which are between those of the clean and adversarially trained models and claim that SGR improves on the clean accuracy for CIFAR-10. But in the table the gap between FGSM and GN/SGR clean accuracies seem fairly small with FGSM providing better robustness (for most source attacks). Even more concerning, is the fact that GN seems to outperform SGR. I do not find these results substantial enough to motivate SGR as a robustness defense compared with adversarial training (or even GN), especially as SGR has the same computational limitations involved with expensive adversarial perturbations.
I felt that the study into the covariance structure of adversarial perturbations was interesting but as it stands was not complete enough to be informative in general. In the conclusion the authors write that they provide evidence that current adversarial attacks act by perturbing the short-range correlations of signals but this has only been confirmed for unregularized classifiers. Despite these issues, I thought that the paper was well written and hope that the empirical study can be improved and clarified.
Minor comments:
- Section 2.1, set of transformations only introduced briefly then forgotten. Leaving output invariant confused me, as this does not apply to adversarial examples.
- Section 2.3, second paragraph l3: In Maaten et al. should be citet.
- Section 3.1, should  make clear that derivative is with respect to the data.
- Section 3.1, define delta as the Hessian clearly (it is used for the simplex in the previous section). Though this is easy to figure out.
- Section 7.1, starts with (iii), is this intentional? Perhaps an introductory sentence could make this clearer.
- Section 7.3, for label leaking, I'm not convinced by this argument alone. Assuming the covariance structure is still computed from a particular adversarial example, I see no compelling reason that this would not occur.
Clarity: The paper is very clearly written and is easy to follow.