DETERMINISTIC PAC-BAYESIAN GENERALIZATION BOUNDS FOR DEEP NETWORKS VIA GENERALIZING NOISE-RESILIENCE
The ability of overparameterized deep networks to generalize well has been linked to the fact that stochastic gradient descent (SGD) finds solutions that lie in flat, wide minima in the training loss -minima where the output of the network is resilient to small random noise added to its parameters. So far this observation has been used to provide generalization guarantees only for neural networks whose parameters are either stochastic or compressed. In this work, we present a general PAC-Bayesian framework that leverages this observation to provide a bound on the original network learned -a network that is deterministic and uncompressed. What enables us to do this is a key novelty in our approach: our framework allows us to show that if on training data, the interactions between the weight matrices satisfy certain conditions that imply a wide training loss minimum, these conditions themselves generalize to the interactions between the matrices on test data, thereby implying a wide test loss minimum. We then apply our general framework in a setup where we assume that the pre-activation values of the network are not too small (although we assume this only on the training data). In this setup, we provide a generalization guarantee for the original (deterministic, uncompressed) network, that does not scale with product of the spectral norms of the weight matrices -a guarantee that would not have been possible with prior approaches.
The authors demonstrate the generalization bound for deep neural networks using the PAC-Bayesian approach. They adopt the idea of noise resilience in the analysis and obtain a result that has improved dependence in terms of the network dimensions, but involves parameters (e.g., pre-activation) that may be large potentially. 
My major concern is also regarding the dependence on the pre-activation that can be very large in practice. This is also shown in the numerical experiments. Therefore, the overall generalization bound can be larger than existing results, though the later have stronger dependence on the network sizes. By examining the analysis for the main result, it seems to me that the reason the authors can induce weaker dependence on network sizes is essentially they involved the pre-activation parameters. This can be viewed as a trade-off how strong the generalization bound depend on the network sizes and other related parameters (like the pre-activation here) rather than strictly tighten the error bound from a more refined/structured way. I also suggest that the authors provide the comparison of their bound and existing ones to see the quantitative difference of the results. 
Regarding the noise resilience, it is not clear to where the noise resilience shows up from the analysis or the result. From the proof of the main result, the analysis seems to be standard as in the PAC-Bayesian analysis, which is based on bounding the difference of the network before and after injecting randomness into the parameters. The difference with respect to the previous result due to the different way of bounding such a gap, where the Jacobian, the pre-activation and function output pop up. But this does not explain how well a network can tolerate the noise, either in the parameter space of the data space. This is different with the previous analysis based on the noise resilience, such as [1]. So, the title and the way the authors explain as noise resilience is somewhat misleading. More detailed explanation will help.
[1] Arora et al. Stronger generalization bounds for deep nets via a compression approach.