Published as a conference paper at ICLR 2018 ON UNIFYING DEEP GENERATIVE MODELS
Deep generative models have achieved impressive success in recent years. Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), as powerful frameworks for deep generative model learning, have largely been considered as two distinct paradigms and received extensive independent studies respectively. This paper aims to establish formal connections between GANs and VAEs through a new formulation of them. We interpret sample generation in GANs as performing posterior inference, and show that GANs and VAEs involve minimizing KL divergences of respective posterior and inference distributions with opposite directions, extending the two learning phases of classic wake-sleep algorithm, respectively. The unified view provides a powerful tool to analyze a diverse set of existing model variants, and enables to transfer techniques across research lines in a principled way. For example, we apply the importance weighting method in VAE literatures for improved GAN learning, and enhance VAEs with an adversarial mechanism that leverages generated samples. Experiments show generality and effectiveness of the transfered techniques.
The authors develops a framework interpreting GAN algorithms as performing a form of variational inference on a generative model reconstructing an indicator variable of whether a sample is from the true of generative data distributions. Starting from the ‘non-saturated’ GAN loss the key result (lemma 1) shows that GANs minimizes the KL divergence between the generator(inference) distribution and a posterior distribution implicitly defined by the discriminator. I found the paper IWGAN and especially the AAVAE experiments quite interesting.  However the paper is also very dense and quite hard to follow at times - In general I think the paper would benefit from moving some content (like the wake-sleep part of the paper) to the appendix and concentrating more on the key results and a few more experiments as detailed in the comments / questions below.
Q1) What would happen if the KL-divergence minimizing loss proposed by Huszar (see e.g http://www.inference.vc/an-alternative-update-rule-for-generative-adversarial-networks/) was used instead of the “non-saturated” GAN loss - would the residial JSD terms in Lemma 1 cancel out then?
Q2) In Lemma 1 the negative JSD term looks a bit nasty to me e.g. in addition to KL divergence the GAN loss also maximises the JSD between the data and generative distributions. This JSD term acts in a somewhat opposite direction of the KL-divergence that we are interested in minimizing. Can the authors provide some more detailed comments / analysis on these two somewhat opposed terms - I find this quite important to include given the opposed direction of the JSD versus the KL term and that the JSD is ignored in e.g. section 4.1? secondly did the authors do any experiments on the the relative sizes of these two terms? I imagine it would be possible to perform some low-dimensional toy experiments where both terms were tractable to compute numerically?
Q3) I think the paper could benefit from some intuition / discussion of the posterior term q^r(x|y) in lemma 1 composed on the prior p_theta0(x) and discriminator q^r(y|x). The terms drops out nicely in math however i had a bit of a hard time wrapping my head around what minimizing the KL-divergence between this term and the inference distribution p(xIy). I know this is a kind of open ended question but i think it would greatly aid the reader in understanding the paper if more ‘guidance’ is provided instead of just writing “..by definition this is the posterior.’
Q4) In a similar vein to the above. It would be nice with some more discussion / definitions of the terms in Lemma 2. e.g what does “Here most of the components have exact correspondences (and the same definitions) in GANs and InfoGAN (see Table 1)” mean? 
Q5) The authors state that there is ‘strong connections’ between VAEs and GANs. I agree that both (after some assumptions) both minimize a KL-divergence (table 1) however to me it is not obvious how strong this relation is. Could the authors provide some discussion / thoughts on this topic?
Overall i like this work but also feel that some aspects could be improved: My main concern is that a lot of the analysis hinges on the JSD term being insignificant, but the authors to my knowledge does but provide any prof / indications that this is actually true. Secondly I think the paper would greatly benefit from concentration on fewer topics (e.g. maybe drop the RW topic as it feels a bit like an appendix) and instead provide a more throughout discussion of the theory (lemma 1 + lemma 2) as well as some more experiments wrt JSD term.