We study the problem of defending deep neural network approaches for image classification from physically realizable attacks. First, we demonstrate that the two most scalable and effective methods for learning robust models, adversarial training with PGD attacks and randomized smoothing, exhibit limited effectiveness against three of the highest profile physical attacks. Next, we propose a new abstract adversarial model, rectangular occlusion attacks, in which an adversary places a small adversarially crafted rectangle in an image, and develop two approaches for efficiently computing the resulting adversarial examples. Finally, we demonstrate that adversarial training using our new attack yields image classification models that exhibit high robustness against the physically realizable attacks we study, offering the first effective generic defense against such attacks.
[Note: I gave a 3 for thoroughness even though I only read the paper once, because I believe that I carefully considered the paper while reading it.]
This paper argues that threat models such as L-inf are limited when considering physically realizable attacks, and provides evidence for this by showing that L-inf adversarial training is insufficient to fully confer robustness against physically realizable attacks in the literature such as the adversarial glasses attack. The paper then proposes an alternate threat model based on contiguous rectangular regions, and shows that adversarial training against this model does far better.
Overall this is a strong paper with thorough experiments, and was for the most part carefully written (although see some quibbles below). I particular liked the paper's carefully distinction between physically *realizable* and physically *realized* attacks, and the admission that not all realizable attacks would fall under the threat model (while still justifying why the model is interesting).
I am on the border of weak accept and strong accept. The main two points keeping me from strong accept are discussed below (and seem perhaps addressable by the authors):
1. The assertion that rectangular occlusions might be a fruitful model for realizable attacks is only lightly tested, since the attacks considered in this paper all fall into the rectangular occlusions threat model. Since one of the key points in the paper is that training in the L-inf threat model could leave vulnerabilities to non-Linf attacks, we might expect the same with rectangular occlusions. A more convincing demonstration would be to test robustness to physically realizable attacks that fall outside the rectangular model (perhaps even skewed or rotated rectangles, although a less synthetic example would be even better). A more minor point is that it would be nice to test the robustness of models trained on rectangular occlusions against L-inf adversarial attacks. This is relevant to knowing whether training on occlusions is giving up on adversarial robustness or actually helps against both (I actually think it's plausible that you would do well on L-inf, but it seems worth testing either way).
2. The high-level claimed take-away that adversarial training does not help against physically realizable attacks seems false in light of Figure 2, which shows that L-inf adversarial training substantially improves robustness relative to the baseline. A more accurate take-away would be that on some datasets adversarial training helps but still leaves a gap, while on others it does not help at all and perhaps hurts. I would prefer more careful wording as a reader who only goes through the introduction might not see Figure 2.
On #1, I should stress that the paper would still be interesting regardless of the outcome of the two experiments in #1; I just think that for thoroughness it would be nice to include them. The existing experiments are already thorough so this would be going above and beyond, but that is why it might help raise my score from weak to strong accept.
EDITED TO ADD: For #2, I would also be happy with an argument from the authors as to why the current language appropriately describes the experiments. I do not wish to dictate to the authors what their take-aways are, but more to open up for discussion a point that seemed slightly sloppy to me.
Minor comments:
Please avoid subjective intensifiers: "We then use an extensive experimental evaluation to demonstrate that our proposed approach is far more robust against physical attacks on deep neural networks than adversarial training and randomized smoothing methods that leverage lp-based attack models." Both "extensive" and "far" are unnecessary.
Make sure to use \citep vs \citet correctly.
First sentence of 2.1 is too verbose. Overall the prose in that paragraph is turgid, due to too many action phrases being turned into nouns. E.g. "The focus is on", "The typical goal is" both indicate *action* and could be profitably turned into verbs. See Williams and Bizup's book on Style.
"since our ultimate goal is to defend against physical attacks, untargeted attacks that aim to maximize error are the most useful" This seems weak; I don't understand why an attack being physical should go in line with it being untargeted; oftentimes an attacker will have a specific targeted goal.
"another advantage of the ROA attack is that it is, in principle,easier to compute than,  say,l∞-bounded attacks" This seems incongruous with the subsequent text, which admits that ROA if implemented naively would be slower than L-inf attacks.