Wasserstein GAN(WGAN) is a model that minimizes the Wasserstein distance between a data distribution and sample distribution. Recent studies have proposed stabilizing the training process for the WGAN and implementing the Lipschitz constraint. In this study, we prove the local stability of optimizing the simple gradient penalty µ-WGAN(SGP µ-WGAN) under suitable assumptions regarding the equilibrium and penalty measure µ. The measure valued differentiation concept is employed to deal with the derivative of the penalty terms, which is helpful for handling abstract singular measures with lower dimensional support. Based on this analysis, we claim that penalizing the data manifold or sample manifold is the key to regularizing the original WGAN with a gradient penalty. Experimental results obtained with unintuitive penalty measures that satisfy our assumptions are also provided to support our theoretical results.
In the paper, WGAN with a squared zero centered gradient penalty term w.r.t. to a general measure is studied. Under strong assumptions, local stability of a time-continuous gradient ascent/descent dynamical system near an equilibrium point are proven for the new GP term. Experiments show comparable results to the original WGAN-GP formulation w.r.t. FID and inception score.
Overall, I vote for rejecting the paper due to the following reasons:
- The proven convergence theorem is for a time-continuous "full-batch" dynamical system, which is very far from what happens in practice (stochastic + time discrete optimization with momentum etc). I don't believe that one can make any conclusions about what is actually happening for GANs from such an idealized setting. Overall, I don't understand why I should care about local stability of that dynamical system.
- Given the previous point I feel the authors draw too strong conclusions from their results. I don't think Theorem 1 gives too many insights about the success of gradient penalty terms.
- There are only marginal improvements in practice over WGAN-GP when using other penalty measures. 
Further remarks:
- In the introduction it is claimed that mode collapse is due to JS divergence and "low-dimensionality of the data manifold". This is just a conjecture and the statement should be made more weak.
- The preliminaries on measure theory are unnecessarily complicated (e.g. partly developed in general metric spaces). I suggest that the authors try to simplify the presentation for the considered case of R^n and avoid unnecessarily complicated ("mathy") definitions as they distract from the actual results. 
==after rebuttal==
After reading the authors rebuttal I increased the my rating to 6 as they addressed some of my doubts. I still think that the studied setting is too idealized, but it is a first step towards an analysis.