One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing decision surfaces in both parameter space and input space, we instead show that the geometry property of decision surface in input space correlates well with the adversarial robustness. We then propose an adversarial robustness indicator, which can evaluate a neural network's intrinsic robustness property without testing its accuracy under adversarial attacks. Guided by it, we further propose our robust training method. Without involving adversarial training, our method could enhance network's intrinsic adversarial robustness against various adversarial attacks.
This paper argues that analyzing loss surfaces in parameter space for the purposes of evaluating adversarial robustness and generalization is ineffective, while measuring input loss surfaces is more accurate. By converting loss surfaces to decision surfaces (which denote the difference between the max and second highest logit), the authors show that all adversarial attack methods appear similar wrt the decision surface. This result is then related to the statistics of the input Jacobian and Hessian, which are shown to differ across adversarially sensitive and robust models. Finally, a regularization method based on regularizing the input Jacobian is proposed and evaluated. All of these results are shown through experiments on MNIST and CIFAR-10.
In general, the paper is clear, though there are a number of typos. With respect to novelty, some of the experiments are novel, but others, including the improved training method, have been explored before (see specific comments for references). Finally, regarding significance, many of the insights provided this paper are true by definition, and are therefore unlikely to have a significant impact. 
While I strongly believe that rigorous empirical studies of neural networks are essential, this paper is lacking in several key areas, including framing, experimental insights, and relation to prior work, and is therefore difficult to recommend. Please see the comments below for more detail. 
Major comments:
1) In the beginning of the paper, adversarial robustness and generalization are equated. However, adversarial robustness and generalization are not necessarily equivalent, and in fact, several papers have provided evidence against this notion, showing that adversarial inputs are likely to be present even for very good models [5, 6] and that adversarially sensitive models can often generalize quite well [2]. Moreover, all the experiments within the paper only address adversarial robustness rather than generalization to unperturbed samples.
2) One of the main results of this paper is that the loss surface wrt input space is more sensitive to adversarial perturbations than the loss surface wrt parameter space. Because adversarial inputs are defined in input space, by definition, the loss surface wrt to the input must be sensitive to adversarial examples. This result therefore appears true by definition. Moreover, [3] related the input Jacobian to generalization, finding a similar result, but is not discussed or cited.
3) The main result of Section 3 is that all adversarial attacks “utilize the decision surface geometry properties to cross the decision boundary within least distance.” While to my knowledge the decision surface visualization is novel and might have important uses, this statement is again true by definition, given that adversarial attack methods try to find the smallest perturbation which changes the network decision. As a result, all methods must find directions which are short paths in the decision surface. It is therefore unclear what additional insight this analysis presents. 
4) How does measuring the loss landscape as an indicator for adversarial robustness differ from simply trying to find adversarial examples as is common? If anything, it seems it should be more computationally expensive as points are sampled in a grid search vs optimized for. 
5) The proposed regularizer for adversarial robustness, based on regularizing the input Jacobian, is very similar to what was proposed in [1], yet [1] is not discussed or cited. 
Minor comments:
1) The paper’s first sentence states that “It is commonly believed that a neural network’s generalization is correlated to ...the flatness of the local minima in parameter space.” However, [4] showed several years ago that the local minima flatness can be arbitrarily rescaled and has been fairly influential. While [4] is cited in the paper, it is only cited in the related work section as support for the statement that local minima flatness is related to generalization when this is precisely opposite the point this paper makes. [4] should be discussed in more detail, both in the introduction and the related work section.
2) The paper is quite lengthy, going right up against the hard 10 page limit. While this may be acceptable for papers with large figures or which require the extra space, this paper does not currently meet that threshold. 
3) Throughout the figures, axes should be labeled. 
4) In section 2.2, it is stated that both networks achieve optimal accuracy of ~90% on CIFAR-10. This is not optimal accuracy and hasn’t been for several years [7].
5) Why is equation 2 calculated with respect to the logit layer vs the normalized softmax layer? Using the unnormalized logits may introduce noise due to scaling.
6) In Figure 8, the scales of the Hessian are extremely different. Does this impact the measurement of sparseness? 
Typos:
1) Introduction, second paragraph: “For example, ResNet model usually converges to…” should be “For example, ResNet models usually converge to…”
2) Introduction, second paragraph: “...defected by the adversarial noises...” should be “...defected by adversarial noise…”
3) Introduction, third paragraph: “...introduced by adversarial noises...” should be “...introduced by adversarial noise…”
4) Section 3.1, first paragraph: “cross entropy based loss surface is…” should be “cross entropy based loss surfaces is…”
[1] Jakubovitz, Daniel, and Raja Giryes. "Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization." arXiv preprint arXiv:1803.08680 (2018). ECCV 2018.
[2] Zahavy, Tom, et al. "Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms." arXiv preprint arXiv:1602.02389 (2016). ICLR Workshop 2018
[3] Novak, Roman, et al. "Sensitivity and generalization in neural networks: an empirical study." arXiv preprint arXiv:1802.08760 (2018). ICLR 2018.
[4] Dinh, Laurent, et al. "Sharp Minima Can Generalize For Deep Nets." International Conference on Machine Learning. 2017.
[5] Fawzi, Alhussein, Hamza Fawzi, and Omar Fawzi. "Adversarial vulnerability for any classifier." arXiv preprint arXiv:1802.08686 (2018). NIPS 2018.
[6] Gilmer, Justin, et al. "Adversarial spheres." arXiv preprint arXiv:1801.02774 (2018). ICLR Workshop 2018.
[7] http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html