Skip connections are an essential component of current state-of-the-art deep neural networks (DNNs) such as ResNet, WideResNet, DenseNet, and ResNeXt. Despite their huge success in building deeper and more powerful DNNs, we identify a surprising property of skip connections in this paper. Use of skip connections allows easier generation of highly transferable adversarial examples. Specifically, in ResNet-like (with skip connections) neural networks, gradients can backpropagate through either skip connections or residual modules. We find that using more gradients from the skip connections rather than the residual modules according to a decay factor, allows one to craft adversarial examples with high transferability. Our method is termed Skip Gradient Method (SGM). We conduct comprehensive transfer attacks against 10 state-of-the-art DNNs including ResNets, DenseNets, Inceptions, Inception-ResNet, Squeeze-and-Excitation Network (SENet) and robustly trained DNNs. We show that employing SGM on the gradient flow can greatly improve the transferability of crafted attacks in almost all cases. Furthermore, SGM can be easily combined with existing black-box attack techniques, and obtain high improvements over state-of-the-art transferability methods. Our findings not only motivate new adversarial research into the architectural characteristics of DNNs, but also open up further challenges for the design of secure DNN architectures.
Summary: 
This paper proposes a modification to standard Projected Gradient Descent to improve transferability of adversarial examples, when the source model is a ResNet-like model containing skip connections. The method, Skip Gradient Method (SGM) modifies the backwards pass to scale down the gradient computed in each residual branch of the model, before these gradients are combined with the gradient from the skip connection. This thus upweights the gradients from the skip connections as opposed to residual modules. The paper demonstrates significant improvements in the single-model black-box transfer setting, against a variety of undefended and defended models.
Strengths:
- Lots of interesting empirical results here! One result which stands out to me is Table 4, where across a wide range of target models, adding SGM to existing techniques cuts defender accuracy by ~1/2 (e.g. 79.9% to 89.66% for SE154). The results in Table 3, showing that even without additional techniques, SGM results in large improvements and outperforms previous approaches, are also quite nice. 
- It's notable that such a simple approach leads to significant improvements.
- Results are very clearly presented, and writing is clear throughout. 
Suggestions for improvement:
I have 3 major concerns: (1) discrepancies between baselines and previously published results (2) unrealistic threat model (3) framing / conclusions drawn by paper.
I suspect (1) is easily addressed, but was not clear to me from the current paper.
(1) Baselines: 
- For multi-step transfer against undefended models (Table 3), the attack success rates seem low compared to numbers reported in e.g. Liu et al. For example, using ResNet-152 as the source, and VGG-19 as the target, Liu et al reports 19% defender accuracy = 81% attacker success. This is significantly higher than the 65.52% reported for MI (both are non-regularized optimization attacks), also stronger than the 80.68% reported for SGM. In general, considering these are *untargeted* attacks, with eps=16, the transfer rates seem pretty low.
- For multi-step transfer against defended models (Table 5), the numbers are slightly lower than previously reported. E.g. attacking IncV3_ens3, Dong et al 19 reports 46.9% for MI, but here, 44.28% is reported. (I realize these differences are slight.)
- In general, it would be nice if the paper were structured so as to make these comparisons easier. For example, the appendix could include tables comparing the baselines reported here, to values previously reported, and explain any discrepancies. Particularly with black-box transfer, where baseline performance is so sensitive to small choices, it's important to ensure baselines are properly implemented, and the current writing of the paper makes it impossible for the reader to assess this unless they are very familiar with the black-box transfer literature.
(2) Unrealistic threat model:
- For black-box transfer, all the strongest attacks use multiple source models (see e.g. NeurIPS 2017 Adversarial Examples contest, the baselines cited in the paper). While this paper shows significant improvements in the single-source setting, these results are significantly weaker than any multiple-source attack. For instance, Liu et al 17 achieve near 100% untargeted success (and near 100% targeted success) against all undefended models they study (many which overlap), and Dong et al 19 report ~85% accuracy against IncV3_ens3 (compared to <60% here).
- There's no reason in practice that an adversary would not employ an ensemble-based attack if they wanted to fool an unknown model.
(3) Framing / conclusions:
- The paper frames the results as a "security vulnerability of ResNets," but the results don't show this. In particular, they show that ResNets make effective *source* models to be used by *attackers*, but they don't imply which models defenders should use in order to be robust to black-box attacks. In this way, the main message of the paper seems misleading to practitioners.
- The main message of the paper thus seems to be "on the transferability of adversarial examples generated  with resnets" as opposed to the "security of skip connections." I would encourage rewriting of the title/intro/conclusion as such.
Overall, there are several very interesting empirical findings in this paper. I view the two main impacts these results could later have would be leading to (1) improved understanding of what causes transfer of adversarial examples and (2) improved understanding of ResNet-like architectures (for example, the results provide some support for the view of ResNets as ensembles of shallow models, cited in the paper). If the paper were written with this view, then concern (2) above becomes unimportant.
Suggestions I believe could strengthen the paper, but I do not view as weaknesses, or necessities:
- Can you decouple the effect of SGM on optimization and transfer? The paper does a bit of this, but restricts the analysis to the single-step case. For instance, if the main effect of SGM is on optimization, this has interesting implications for optimization of ResNets. If the main effect is on transfer, this has interesting implications for what causes transfer.
Minor:
- I'd suggest moving Table 1 (one-step attacks) to the Appendix. The results are less impressive than multi-step, and the community as a whole favors stronger attacks.
Overall, there are several interesting empirical findings in this paper (modulo concerns about baselines indicated above). I suggest that the paper either consider more realistic threat models to be useful to the adversarial examples field, or focus on the insights revealed by these findings. I hope that the authors can address the concerns outlined here, and I would be happy to adjust my score if so.
Note that my current indicated confidence rating is for the current state of the paper. I would be happy to adjust my evaluation if revisions to the paper can one/several of the concerns indicated above.
__________________
EDIT: Rating adjusted from 3: Weak Reject to 8: Accept, after accounting for revisions and author rebuttal. See reply below.