The problem of adversarial examples has shown that modern Neural Network (NN) models could be rather fragile. Among the most promising techniques to solve the problem, one is to require the model to be -adversarially robust (AR); that is, to require the model not to change predicted labels when any given input examples are perturbed within a certain range. However, it is observed that such methods would lead to standard performance degradation, i.e., the degradation on natural examples. In this work, we study the degradation through the regularization perspective. We identify quantities from generalization analysis of NNs; with the identified quantities we empirically find that AR is achieved by regularizing/biasing NNs towards less confident solutions by making the changes in the feature space (induced by changes in the instance space) of most layers smoother uniformly in all directions; so to a certain extent, it prevents sudden change in prediction w.r.t. perturbations. However, the end result of such smoothing concentrates samples around decision boundaries, resulting in less confident solutions, and leads to worse standard performance. Our studies suggest that one might consider ways that build AR into NNs in a gentler way to avoid the problematic regularization.
===== Summary ===== 
The paper presents new theory to develop understanding about why adversarially robust neural networks show lower test performance compared to their standard counterparts despite being more robust to perturbations in the data. The main hypothesis is that the degradation in performance in adversarially robust networks is due to many samples being concentrated around the decision boundary, which makes the network less confident about its decisions. The paper studies this hypothesis by deriving a bound on the generalization error based on the margin between the samples in the training set and the decision boundary. The paper then presents empirical demonstrations that aim to illustrate the theoretical findings. 
Contributions:
1. Derive a generalization bound on the performance of adversarially robust networks  that depends on the margin between training examples and the decision boundary.
2. Provide empirical evaluations that aim to illustrate the theoretical results.
===== Review =====
The problem that the paper addresses is very significant to the robust optimization field and the study of adversarial robustness in neural networks. Thus, I believe that the results could represent a significant contribution. However, due to the way that the information is presented, it is difficult to validate the correctness of the theory and the insights from the paper. Consequently, I consider that the paper should be rejected.
===== Detailed Comments =====
- First, and foremost, the paper should be proof-read for English grammar and writing style. In its current form, it is difficult to follow the main argument of many of the paragraphs. This is exceedingly important because the main subject of the paper is already difficult to digest as is.
- In the related work section, a lot of previous work is referenced without any context about what the contribution of each of those papers is. Each of these papers should be mentioned along with their corresponding contributions. Otherwise, it is difficult to frame the paper within the context of the current literature. Moreover, not providing context makes it difficult to determine which parts of the paper are original and which are the result from previous work. 
- The first item in Section 1.1 is difficult to follow because there are many gaps in logic that are left to the reader to fill in. It is reasonable to expect the reader to fill in some of the details, but since the sole purpose of this section is to build intuition about what is about to be presented in the paper, then each step in the explanation should follow as seamlessly as possible.
- The motivation for studying the margins between the training set and the decision boundary is not clear until Section 5.2.1 where it is mentioned that this is a widely used tool in learning theory. This should be presented earlier since not every reader will be completely familiar with learning theory. Moreover, it would also be useful to provide some intuition about how margins relate to the confidence of a classifier.
- After presenting the main result of the paper — Theorem 4.1 — very little intuition is provided about each of the terms in the bounds. It would greatly increase the clarity of the result if each term was explained intuitively, so that the readers can gain the main insight of the paper before reading the proofs. This would also help motivate better the empirical evaluations in the following section. 
- The plots in FIgure 3 and Figure 4 are very difficult to understand and unintuitive. For Figure 3, the main reason the plots are difficult to read is because different colors are used for different networks. For Figure 4, it is difficult to understand what is happening because the tallest curves are plotted on top of the shortest ones. Hence, the information from the other curves is mostly lost. This seems like a very minor comment, but this graphs are not very complicated, so they should be easy to understand; yet it takes several minutes to take in what is happening in the graph.
- For the proof of Lemma 4.1it is not clear how to get from Equation (7) to the main result of the lemma even after referencing Theorem 3 of Sokolic et. al. (2017) and Jia et. al. (2019). This could also be my lack of expertise on the topic; however, since the proof is already in the appendix, the proof should not be sparse in the amount of detail that it provides. 
===== References =====
Jure Sokolic, Raja Giryes, Guillermo Sapiro, and Miguel R. D. Rodrigues. Robust Large Margin Deep Neural Networks. IEEE Transactions on Signal Processing, 65(16):4265–4280, aug 2017. ISSN 1053-587X. doi: 10.1109/TSP.2017.2708039.
Kui Jia, Shuai Li, Yuxin Wen, Tongliang Liu, and Dacheng Tao. Orthogonal Deep Neural Networks. Technical report, 2019. URL http://arxiv.org/abs/1905.05929.