Robustness and Equivariance of Neural Networks
Neural networks models are known to be vulnerable to geometric transformations as well as small pixel-wise perturbations of input. Convolutional Neural Networks (CNNs) are translation-equivariant but can be easily fooled using rotations and small pixel-wise perturbations. Moreover, CNNs require sufficient translations in their training data to achieve translation-invariance. Recent work by Cohen & Welling (2016), Worrall et al. (2016), Kondor & Trivedi (2018), Cohen & Welling (2017), Marcos et al. (2017), and Esteves et al. (2018) has gone beyond translations, and constructed rotation-equivariant or more general group-equivariant neural network models. In this paper, we do an extensive empirical study of various rotation-equivariant neural network models to understand how effectively they learn rotations. This includes Group-equivariant Convolutional Networks (GCNNs) by Cohen & Welling (2016), Harmonic Networks (H-Nets) by Worrall et al. (2016), Polar Transformer Networks (PTN) by Esteves et al. (2018) and Rotation equivariant vector field networks by Marcos et al. (2017). We empirically compare the ability of these networks to learn rotations efficiently in terms of their number of parameters, sample complexity, rotation augmentation used in training. We compare them against each other as well as Standard CNNs. We observe that as these rotation-equivariant neural networks learn rotations, they instead become more vulnerable to small pixel-wise adversarial attacks, e.g., Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD), in comparison with Standard CNNs. In other words, robustness to geometric transformations in these models comes at the cost of robustness to small pixel-wise perturbations.
 This paper empirically studies various CNN robustifying mechanisms aiming to achieve rotational invariance. The main finding is that such robustifying mechanisms may lead to lack of robustness against pixel-level attacks such as FGSM and its variants. The paper does a comprehensive job in studying relevant robustifying schemes and attacks strategies. However, the paper does not present sufficiently new information worthy of a regular conference paper, it can be a good workshop paper though for the Robust Learning community. Some analytical insights would really strengthen the work. Also, from an empirical standpoint, the authors need to consider other data sets beyond just the MNIST data set. xxxxxxxxxxxxxx While I appreciate the authors* rebuttal and revisions, I still do not see sufficient contribution here worthy of a regular ICLR paper.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Robustness and Equivariance of Neural Networks
Neural networks models are known to be vulnerable to geometric transformations as well as small pixel-wise perturbations of input. Convolutional Neural Networks (CNNs) are translation-equivariant but can be easily fooled using rotations and small pixel-wise perturbations. Moreover, CNNs require sufficient translations in their training data to achieve translation-invariance. Recent work by Cohen & Welling (2016), Worrall et al. (2016), Kondor & Trivedi (2018), Cohen & Welling (2017), Marcos et al. (2017), and Esteves et al. (2018) has gone beyond translations, and constructed rotation-equivariant or more general group-equivariant neural network models. In this paper, we do an extensive empirical study of various rotation-equivariant neural network models to understand how effectively they learn rotations. This includes Group-equivariant Convolutional Networks (GCNNs) by Cohen & Welling (2016), Harmonic Networks (H-Nets) by Worrall et al. (2016), Polar Transformer Networks (PTN) by Esteves et al. (2018) and Rotation equivariant vector field networks by Marcos et al. (2017). We empirically compare the ability of these networks to learn rotations efficiently in terms of their number of parameters, sample complexity, rotation augmentation used in training. We compare them against each other as well as Standard CNNs. We observe that as these rotation-equivariant neural networks learn rotations, they instead become more vulnerable to small pixel-wise adversarial attacks, e.g., Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD), in comparison with Standard CNNs. In other words, robustness to geometric transformations in these models comes at the cost of robustness to small pixel-wise perturbations.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper empirically studies various CNN robustifying mechanisms aiming to achieve rotational invariance. The main finding is that such robustifying mechanisms may lead to lack of robustness against pixel-level attacks such as FGSM and its variants. The paper does a comprehensive job in studying relevant robustifying schemes and attacks strategies. However, the paper does not present sufficiently new information worthy of a regular conference paper, it can be a good workshop paper though for the Robust Learning community. Some analytical insights would really strengthen the work. Also, from an empirical standpoint, the authors need to consider other data sets beyond just the MNIST data set. xxxxxxxxxxxxxx While I appreciate the authors* rebuttal and revisions, I still do not see sufficient contribution here worthy of a regular ICLR paper.
assistant
 This paper empirically studies various CNN robustifying mechanisms aiming to achieve rotational invariance. The main finding is that such robustifying mechanisms may lead to lack of robustness against pixel-level attacks such as FGSM and its variants. The paper does a comprehensive job in studying relevant robustifying schemes and attacks strategies. However, the paper does not present sufficiently new information worthy of a regular conference paper, it can be a good workshop paper though for the Robust Learning community. Some analytical insights would really strengthen the work. Also, from an empirical standpoint, the authors need to consider other data sets beyond just the MNIST data set. xxxxxxxxxxxxxx While I appreciate the authors* rebuttal and revisions, I still do not see sufficient contribution here worthy of a regular ICLR paper.