Generative adversarial networks (GANs) have made significant progress on realistic image generation but often suffer from instability during the training process. Most previous analyses mainly focus on the equilibrium that GANs achieve, whereas a gap exists between such theoretical analyses and practical implementations, where it is the training dynamics that play a vital role in the convergence and stability of GANs. In this paper, we directly model the dynamics of GANs and adopt the control theory to understand and stabilize it. Specifically, we interpret the training process of various GANs as certain types of dynamics in a unified perspective of control theory, which enables us to model the stability and convergence easily. Borrowed from control theory, we adopt the widely-used negative feedback control to stabilize the training dynamics, which can be considered as an L2 regularization on the output of the discriminator. We empirically verify our method on both synthetic data and natural image datasets. The results demonstrate that our method can efficiently stabilize the dynamics and converge to better results.
This paper gives a perspective of GANs from control theory, where it is well established in which cases a dynamical system (which can be described analytically as a function of time) is stable or not (by looking at the roots of the denominator of the so-called transfer function in control theory). It is interesting that the analysis using this framework on simple examples is in line with known results in the GAN literature (Dirac GAN). 
Although I personally enjoyed reading the results that from control theory perspective are inline with GAN literature, the paper does not provide novel surprising results. For e.g. the results on the oscillating behavior of Dirac-GAN are described in related works (e.g. Mescheder et al. 2018), and in practice, WGAN with no regularization is not used (as well as GAN with momentum, as normally beta1=0 in practice). In my understanding, the authors present these results to justify the validity of the approach. However, this limits the novelty of the results relative to existing literature. The authors do not focus (in the main paper) on GAN variants used currently, and it is not clear if the proposed approach provides improvement relative to the current state of the art implementations (see next paragraph). Moreover, if I understand correctly the WGAN analysis does not take into account that G and D are non-linear, and it is unclear if these can be done.
I am also wondering if the comparison with the baselines is fair. In other words, although in the present results, the proposed NF-SGAN/WGAN outperforms the baseline, the reported performance of the baselines is worse than in related works on CIFAR10. In particular, FID of ~30 on CIFAR10 for the baselines is notably higher then current reported results on this dataset (e.g. Miyato et al. 2018; Chavdarova et al. 2019). In my opinion, the authors could start from the existing state of the art implementations on this dataset, and report if negative feedback (NF) improves upon.
As the approach uses NF is derived specifically for unstable dynamics, it is not clear to me how adding it would affect the training if the dynamics *is stable*. In this context, I consider that for example, the work by Balduzzi et al. 2018 may be relevant as it describes that the dynamics of games (the Jacobian) has two components, one of which describes the oscillating behavior (Hamiltonian game); whereas most games are a mix of oscillating and non-oscillating dynamics. It is not clear to me if NF would improve stability/performances in general games. As the authors’ main claim is improved stability I am curious to see more detailed analysis on real-world datasets (e.g. multiple seed runs, 2nd-moment estimates over iterations as in Chavdarova et al. 2019).
In summary, although the proposed perspective seems promising given the presented results and it is interesting, in my opinion, it does not provide novel insights nor obtains current state-of-the-art results on CIFAR10, or guarantee that if pursuing it would allow for solving current issues of GAN training. 
--- Minor ---
- Abstract: ‘converge better’ it is not clear to me in what sense (faster/better final performances)
- Page 3, Eq. 5: as D and G are functions of time here, eq 5 should maybe be written in more detail to include t
-  Page 3, Sec. 3: I think citing works that also focus on Dirac-GAN would motivate better why you focus on Dirac-GAN in this paper (e.g. writing `as in Mescheder et al. 2018`)
- Page 4: infinity - infinite
- Page 5: can also.. explains ->  explain