New estimates for the generalization error are established for a nonlinear regression problem using a two-layer neural network model. These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model. In contrast, most existing results for neural networks are a posteriori in nature in the sense that the bounds depend on some norms of the model parameters. The error rates are comparable to that of the Monte Carlo method in terms of the size of the dataset. Moreover, these bounds are equally effective in the over-parametrized regime when the network size is much larger than the size of the dataset.
This paper studies the generalization properties of a two layer neural network for a nonlinear regression problem where the target function has a finite spectral norm. The generalization bound comprises of an approximation term (dependent on the width) and an estimation term (dependent on spectral norm of the target and scaling as 1/sqrt{n}). 
The key contribution is the derivation of the generalization bound where the estimation term depends on the properties of the target function rather than properties of the class of two layer neural networks. These bounds are instantiated for a class of regularized estimators (with path norm regularization).
1. Theoretical Novelty: While both Theorem 3 (approximation) and Theorem 4 (a posterior generalization) were mostly following known results, the key development seems to be the bound on the path norm of the regularized solution in terms of the spectral norm of the target function. Given that the estimator is a path-norm regularized estimator, this seemed to be an incremental contribution. What would be more interesting is to obtain such a bound for an unregularized estimator: either saying something about the optimization procedure or relating this kind of regularization to properties of the dataset over which it is trained.
2. Regression vs Classification: While the focus of the paper is on a regression problem, the experiments and problem motivation seems to arise from a classification setting. This creates a mismatch between the what the paper is about and the problem that has been motivated. Would it be possible to extend these results to loss functions (other than squared loss ) like cross-entropy loss or hinge loss which indeed work in the classification setting?
3. Comparison with Klusowski & Barron (2016): In the comparison section, it is mentioned that  Klusowski & Barron (2016) analyze a "similar" problem and obtain worse generalization bounds. It would be important to know the exact setting in which they obtained their bounds and how do their assumptions compare with the ones made in this paper. The comparison seems incomplete without this.
4. The experiments showcase that the regularized estimator has a better path norm (and expectedly so) but almost similar (in case of MNIST actually better) test accuracy. This defeats the purpose of  showing that the regularized estimator has better generalization ability which is claimed in the introduction as well as the experiment section (calling it "well-posed"). What this indeed shows is that even though the path norm might be big, the generalization of the estimator is till very good contradicting the statements made. 
5. The numbers shown in Figure 1 and the numbers reported in Table 2 do no match: while the plot shows that the scaled path norm is around 60 for both MNIST and CIFAR-10, the corresponding numbers in the table are 507 and 162. Can you please point out the reason for this discrepancy?
6. Theorem 5 seems to suggest that in the noiseless case, the estimation error would scale as the spectral norm of f^*. Rather, in the noiseless setting, it seems that the correct scaling of the generalization error should be with respect to properties of the regularized estimator and the function class. Even though the spectral norm can be arbitrarily high, the generalization bound should only be dependent on the complexity of functions which can fit the current data well. It would be good to have a comment in the draft on why the current dependence is a better thing and examples where such generalization bounds are indeed better.