Many defenses for Convolutional Neural Networks are based on a simple observation that the adversarial examples are not robust and small perturbations to the attacking input often recover the desired prediction. Intuitively, the adversarial examples occupy a very small cone in the decision space which is surrounded by a large area corresponding to the correct class. While the intuition is simple, a detailed understanding of this phenomenon is missing from the research literature. We identify a family of defense techniques that are based on the instability assumption. The defenses include deterministic lossy compression algorithms and randomized perturbations to the input that all lead to similar gains in robustness. We present a comprehensive experimental analysis of when and why perturbation defenses work and potential mechanisms that could explain their effectiveness (or ineffectiveness) in different settings.
The paper studies the robustness of adversarial attacks to transformations of their output. Specifically, for standard methods for crafting adversarial examples, the authors evaluate whether the crafted examples remain adversarial under (stochastic or deterministic) transformations such as Gaussian noise and SVD compression. The authors argue that different transformations have a similar impact on the perturbed inputs. Finally, they argue that the reason why these transformations can sometimes recover the correct label is due to the loss being more unstable at these points (from a first- and second-order pespective).
From a conceptual point of view, I did not find the paper particularly impactful. In particular, I can identify three claimed contributions:
a) The L2 distortion introduced by these transformation might have more impact than the specific transformation used (figure 1).
Firstly, I would argue that this effect is most prominent for small L2 distortions (distortions larger than 5/40 do exhibit noticeable differences). Secondly, I am not sure why one would expect these transformations to have fundamentally different impact. The adversarial attacks considered manipulate the input using first-order methods in input space. Since the transformations are model- and data-agnostic it seems expected that the primary mechanism behind their effect is the pixel-wise distortion of the image.
b) Adversarial perturbations that are robust to one type of transformation tend to also be robust to other transformations (table 2).
This is probably the most interesting observation of the paper, indicating that attackers can bypass several of these transformation defenses by only aiming to be robust to a subset of them. At the same time, I am not sure what the impact of this observation is given that we already know how to bypass most of these defenses anyway.
c) The instability of adversarial perturbations can be explained by a larger gradient norm and Hessian spectrum (figure 2, 3).
First of all, I do not understand how this is considered a potential explanation of empirical behavior. If gradient norm is indicative of instability, then _natural_ images would be unstable (since the gradient wrt the wrong class is large). Furthermore, instability does not explain why adding noise leads to the _correct_ class as opposed to a random class. Perhaps most importantly, from what I understand (also skimming the code), Figure 2 plots the gradient of the _cross-entropy loss_ with respect to each class. However, the norm of the gradient is directly affected by the softmax probability of each class. Hence, for natural images, the probability of the correct class is high leading to a small norm while for the adversarial class it is low leading to large norm. Based on this reasoning, this plot does convey information about the classifier's stability but rather about the softmax probabilites assigned to each class.
In summary, after reading the paper, I am not sure how our understanding of transformation robustness has changed or what insight we have gained that will help us design future attacks and defenses (especially given prior work on how most of these defenses can be bypassed). I thus recommend rejection.
Comments to authors:
-- Many of the attack details are missing (what is the epsilon allowed for PGD? what does the "c" parameter correspond to for CW attacks?) which prevented me from fully comprehending the experimental results. 
-- LBFGS is not a particular attack, it is a general optimization algorithm, https://en.wikipedia.org/wiki/Limited-memory_BFGS 
-- The fact that adversarial points are close to correctly classified points does not mean that a small amount of random noise will change the classifier prediction. In high dimension, the distance to a hyperplane can be epsilon but moving in a random direction requires movement of epsilon * sqrt(number of dimensions) .
==============
UPDATE: I appreciate the authors' response. As stated in my original review, I do recognize the performance of a non-adaptive adversary as a contribution. However, I still don't think that the impact of this finding is significant enough for publication at ICLR. I hence keep my original score.
Response to specific points:
-- Unfortunately, the authors did not address my concerns about the gradient of the loss. The experiments and reasoning of the paper should be sufficient without citing other work in the author response. Moreover, it is unclear if the papers referenced in the response are showing fundamental properties of adversarial examples or are specific to the attacks considered.  For instance, there has been work challenging these papers (https://arxiv.org/abs/1907.12138).
-- As mentioned in my original review, measuring the norm of the gradient of the softmax loss will inevitably take into account the predicted probability of each class. Hence in order to truly measure sensitivity it would be important to work with the logits instead of the softmax probabilities.