This paper focuses on the theoretical investigation of unsupervised generalization theory of generative adversarial networks (GANs). We first formulate a more reasonable definition of general error and generalization bounds for GANs. On top of that, we establish a bound for generalization error with a fixed generator in a general weight normalization context. Then, we obtain a width-independent bound by applying p,q and spectral norm weight normalization. To better understand the unsupervised model, GANs, we establish the generalization bound, which uniformly holds with respect to the choice of generators. Hence, we can explain how the complexity of discriminators and generators contribute to generalization error. For p,q and spectral weight normalization, we provide explicit guidance on how to design parameters to train robust generators. Our numerical simulations also verify that our generalization bound is reasonable.
This paper proves generalization bounds for GANs. I think the paper can be improved significantly in several ways:
1- Writing: The first two sections are relatively well-written. The problem starts at section 3 and continues after that. Some of the things that can be improved:
a) The discussion on the different definitions of generalizations is not really helpful in the current format. You might want to explain how these different definitions relate to each other. For example, if generalization in one of them implies generalization in the other one, etc.
b)Theorem 2.3 is a general statement but it is followed by Corollary 3.3 which is a very specific generalization bound. There is no explanation how one can show the corollary.  Even worse is mixing these two in the proof of the theorem in the appendix.   Please consider improving the use of Theorems, Lemmas and Corollaries.
c) Section 3 and 4 have bunch of theorems and collieries without much explanation. It is not clear that all of these are actually helpful for the main purpose of the paper.
d) I don't completely understand the notation in Corollary 3.3. Eg. what is d_{f,\ell}?
2) Related Work: I think authors need to do a more comprehensive literature review on generalization bounds. Since the generalization bounds presented here are built on the supervised learning bounds, authors discuss the generalization bounds in supervised learning. For example, authors heavily rely on Chen et al. (2019) for their generalization bounds while very similar results where shown before by [1] and [2].
[1] Neyshabur, Behnam, Ryota Tomioka, and Nathan Srebro. "Norm-based capacity control in neural networks." Conference on Learning Theory. 2015.
[2] Golowich, Noah, Alexander Rakhlin, and Ohad Shamir. "Size-independent sample complexity of neural networks." Conference on Learning Theory. 2018.
3) Definition of generalization: I don't think the definition of generalization suggested in this work is much different than Arora et. al. since f really doesn't depend on samples from D_g and hence the empirical and true distributions are not very different. In fact, I think the definition provided by Arora et. al. 2017 is preferred because at the end of the day, we have to estimate the distribution D_g by generating some samples.
4) Generalization bound for fixed g: Unfortunately, the novelty of these generalization bounds are very limited as they are a direct application of known generalization bounds in the supervised settings. Therefore, the authors contributions are very limited here.
5) Generalization bounds for all generators: Again, here the novelty and final result is very limited since the bounds achieved by a union bound arguments and does not really go beyond that.
6) Experiments: Experiments can also be improved significantly. Currently, the correlation is reported for 5 trained networks and it is not clear to me that this result is statistically significant. Moreover, only one hyper-parameter is changed in the experiments which could be problematic. I suggest authors to change multiple hyper-parameters and train more networks to improve the evaluation.
******************************
After author rebuttals:
I have read the authors response and looked at the revision. Unfortunately, many of my concerns are not addressed adequately so my score remains the same.