Adversarial examples are malicious inputs crafted to cause a model to misclassify them. In their most common instantiation, "perturbation-based" adversarial examples introduce changes to the input that leave its true label unchanged, yet result in a different model prediction. Conversely, "invariance-based" adversarial examples insert changes to the input that leave the model's prediction unaffected despite the underlying input's label having changed. So far, the relationship between these two notions of adversarial examples has not been studied, we close this gap. We demonstrate that solely achieving perturbation-based robustness is insufficient for complete adversarial robustness. Worse, we find that classifiers trained to be p -norm robust are more vulnerable to invariance-based adversarial examples than their undefended counterparts. We construct theoretical arguments and analytical examples to justify why this is the case. We then illustrate empirically that the consequences of excessive perturbation-robustness can be exploited to craft new attacks. Finally, we show how to attack a provably robust defense -certified on the MNIST test set to have at least 87% accuracy (with respect to the original test labels) under perturbations of ∞ -norm below ε = 0.4 -and reduce its accuracy (under this threat model with respect to an ensemble of human labelers) to 60% with an automated attack, or just 12% with human-crafted adversarial examples.
================ Update after reading the rebuttal and the revised paper ======================================= 
I have now read the author rebuttal and the revised paper. I had raised two main issues with the paper in my initial review (see below): 1) experiments don't provide enough support for the main claim; 2) the main claim seems to contradict an earlier result by Engstrom et al. (2019). 
With respect to the second point, the authors have responded that Engstrom et al. (2019) use relatively small perturbation sizes during adversarial training and that the trade-off between invariance vs. perturbation based adversarial robustness may be different for such small perturbation sizes. This sounds plausible, although I would have much preferred a more direct demonstration of this (for example, by adversarially training MNIST models with different perturbation sizes, then looking at their performance on invariance-based adversarial examples).
With respect to the first point, the authors have pointed to some numbers in Table 1, but my criticism still stands: for example, the perturbation-robust ABS performs better than the undefended CNN model (similarly for the l_inf PGD model, even though it is trained for a different metric). This seems to contradict the authors' claim that excessive perturbation robustness leads to worse performance on invariance-based adversarial examples.
In conclusion, I very much appreciate the authors' rebuttal and I definitely think there are some interesting ideas in this paper. However, given that one of my main concerns has not been addressed and the other one only insufficiently addressed, I'm inclined to keep my score as it is. Going forward, I would encourage the authors to develop the trade-off between invariance vs. perturbation based adversarial sensitivity for different perturbation sizes more thoroughly.
========================================================================================================
This paper argues that the standard notion of l_p-norm bounded adversarial examples does not adequately capture all types of adversarial examples we may care about. In particular, the authors note the existence of adversarial examples caused by the excessive invariance of classifiers to semantically meaningful perturbations. This was noted in earlier works as well, but the current paper purports to establish a link between l_p-ball robustness and invariance-based adversarial examples. It also introduces a method to generate such invariance-based adversarial examples. There are interesting ideas in this paper, however I have some questions and concerns about some of the claims made in the paper that I would like to see addressed. Here are the main issues for me:
1) If I’m not mistaken, the main claim of the paper, namely that l_p-ball robustness worsens the performance on invariance-based adversarial examples does not actually seem to be supported by any result in the paper beyond some of the simplest, toyest examples.  For example, in the MNIST examples in Table 1, I don’t see how l_p-ball robust models are performing worse than the other models on the invariance-based adversarial examples (the results seem mixed at best). Even in some of the toy examples, this claim is not supported: for example, in the adversarial spheres example, the l_p-ball robust classifier would correspond to the max-margin classifier, which is not vulnerable to invariance-based adversarial examples. Even the ad-hoc sub-optimal classifier chosen in this example doesn't show that l_p-ball robust models are more vulnerable to invariance-based adversarial examples, just that the two phenomena are distinct. There’s the single ImageNet example in Figure 6, which is cute and is consistent with the claim, however it’s just a single example. Also please note that this example does not work with the l_inf norm, so it just shows the inadequacy of the l_2 norm, not of l_p norms in general.
2) The main claim of this paper seems to contradict a result in Engstrom et al. (2019) (https://arxiv.org/abs/1906.00945). They also demonstrate the existence of invariance-based adversarial examples in non-robust models (e.g. see their Figure 2), but their results seem to suggest that these types of examples do not arise in robust models (models trained with adversarial training). Please discuss this paper and clarify the seeming discrepancy between the results there and your claims.
Other issues: 
3) It is really hard to follow the invariance-based adversarial example generation process described in section 4.2. Please describe this more clearly, motivating each step of the process (currently the steps seem ad hoc, it is not explained why each step is needed), so that a reader can understand the generation process at a high level without having to consult the appendix. 
4) The labels are misaligned in Figure 4.
5) In section 4.3, it is mentioned that “we additionally hand-crafted 50 invariance adversarial examples under the specific norms”. It is not explained at all why these additional examples are needed and it is not described how exactly these are generated (neither in the main text, nor in the appendix as far as I can see).
6) In Table 1, the different models are not described at all (neither in the main text, nor in the appendix). They are just acronyms right now. Please describe what these models are.
7) The idea that there may be trade-offs between different notions of robustness and that l_p-ball robustness is not the be all and end all of all robustness measures have been noted in some previous works: for example, Yin et al. (2019): https://arxiv.org/abs/1906.08988 Although the phenomenon discussed in Yin et al. (2019) is different from the one highlighted in the current paper, the main message seems quite similar. So, please discuss this connection in your paper.