Graphical Generative Adversarial Networks
We propose Graphical Generative Adversarial Networks (Graphical-GAN) to model structured data. Graphical-GAN conjoins the power of Bayesian networks on compactly representing the dependency structures among random variables and that of generative adversarial networks on learning expressive dependency functions. We introduce a structured recognition model to infer the posterior distribution of latent variables given observations. We generalize the Expectation Propagation (EP) algorithm to learn the generative model and recognition model jointly. Finally, we present two important instances of Graphical-GAN, i.e. Gaussian Mixture GAN (GMGAN) and State Space GAN (SSGAN), which can successfully learn the discrete and temporal structures on visual datasets, respectively.
This paper proposes Graphical-GAN, a variant of GAN that combines the expressivity of Graphical Models (in particular, Bayesian nets) with the generative inductive bias of Generative Adversarial Networks. Using adversarial learning, the resulting hybrid generative model minimizes for a f-divergence between the joint distributions of the generative model p(X, Z) and an auxiliary encoding distribution q(X,Z). For highly structured latent variables, such as the ones considered in this work, the discriminator's task of distinguishing X,Z samples from the two distributions can be different.   As a second major contribution, the work proposes a learning procedure inspired by Expectation Propogation (EP). Here, the factorization structure of the graphical model is explicitly exploited to make the task of the discriminator "easier" by comparing only subsets of variables. Finally, the authors perform experiments for controlled generation using a GAN model with a mixture of Gaussians prior, and a State-Space structure to empirically validate their approach.  The paper is clearly written, easy-to-follow and achieves a good balance of intuition and formalism. The paper is closely related to prior works at the intersection of inference in GANs, in particular ALI and BiGAN. However, the key novelty of the work is the use of the formalism of graphical models for structured learning and inference. That is, instead of assuming a simple dependence of Z->X, the work looks at richer, alternate factorizations which can explicitly encode prior knowledge, such as a Gausssian Mixture Model.   To the best of my knowledge, the work is technically correct. The empirical evaluation is moderately convincing (shortcomings explained in more detail later in the review). At a broader level, this work asks the following question: Does the structure of graphical models have a role to play in generative modeling? The answer is a clear yes to me, and I was highly pleased to see this work taking a step in this direction. As the authors have noted, the first work in this regime is due to Johnson et al. (done in the context of VAEs) --- however this work is the first to do for adversarially learned generative models.  For the rebuttal phase, I would like to hear the author's responses to the following questions/comments:  1. Since we are using GANs, likelihood evaluation is intractable (except for invertible decoders). Then, the key benefit of Graphical-GANs over other models is controlled generation and inferring latent representations in the presence of prior information (such as the MNIST dataset can be assumed to be generated from 10 independent GANs). InfoGAN enjoys all of these properties. In L199, it is said that GMGAN doesn't need a regularization loss and is heirarchical, which makes it superior. Even though these distinctions are valid, I don't see why the Graphical-GAN approach should be preferred over InfoGAN, both in theory and experiments.   In fact, Graphical-GNN requires additional information tn the form of a strcutured graphical model. On the other hand, InfoGAN seems to automatically uncover this hidden structure and hence, would be more immune to "structure mismatches" between the model and the assumed data. Can the authors comment on this and if possible, report summary experimental results comparing with InfoGAN?  2.  Missing experiments -- Why are the entries in first and third column of Table 1 missing? What about clustering accuracy on CIFAR-10 for the different baselines?  3. The use of the term generalization in L275 is misleading. The only widely accepted definition of generalization I am familiar with posits that the train and test performance of a learned model should be similar to claim that the model is generalizing. This is really the opposite of generalization we see in Figure 8 and lines 275-277 where the training dataset is very different from the generated data in the length of the frames.  4. Looking at the full page of miniature samples in Figure 1 and 2 in Appendix E and Figure 5 in Appendix F tells me nothing about the performance of GMGAN-L on axis of interest. Since these are all labelled/attributed datasets, I would like to see some quantitative metrics such as average clustering accuracy, inter-intra clustering distances etc. to be able to derive a conclusion.  ---- ---- Thanks for the response. I am convinced with the alternate explanation that the proposed framework can solve a diversity of tasks with much richer graphical structures, unlike InfoGAN. This reduces the significance of the MNIST experiments compared to prior work. And R2 has raised some valid concerns with the SSGAN experiments that the authors could address in a later version.