Generative modeling of natural images has been extensively studied in recent years, yielding remarkable progress. Current state-of-the-art methods are either based on maximum likelihood estimation or adversarial training. Both methods have their own drawbacks, which are complementary in nature. The first leads to over-generalization as the maximum likelihood criterion encourages models to cover the support of the training data by heavily penalizing small masses assigned to training data. Simplifying assumptions in such models limits their capacity and makes them spill mass on unrealistic samples. The second leads to mode-dropping since adversarial training encourages high quality samples from the model, but only indirectly enforces diversity among the samples. To overcome these drawbacks we make two contributions. First, we propose a novel extension to the variational autoencoders model by using deterministic invertible transformation layers to map samples from the decoder to the image space. This induces correlations among the pixels given the latent variables, improving over commonly used factorial decoders. Second, we propose a training approach that leverages coverage and quality based criteria. Our models obtain likelihood scores competitive with state-of-the-art likelihood-based models, while achieving sample quality typical of adversarially trained networks.
The paper proposes to alleviate two issues with VAEs and GANs
Mode covering behaviour of the MLE loss used in VAEs causing blurry image samples despite high likelihood scores (Coverage driven training in the paper)
Poor likelihood scores in GAN based models despite great looking amples (Quality driven training in the paper)
Both of these issues are well known and have been previously described in the literature (e.g. MLE models: bishop 2006, GANs: arjovsky et. al. 2017 or sonderby et. al. 2017)
The main contribution of the paper are described in the Eq (8) augmenting the VAE ELBO (L_C term) with a GAN-discriminator based loss (L_Q term). Combining the fact that the L_Q loss (essentially MLE) minimises KL[data|model] and L_Q discriminator loss used minimises KL[model|data] the authors show that this in theory minimizes a lower bound on the forward and reverse KL divergence i.e. somewhere between mode-covering and mode-seeking behavior
As a secondary contribution the authors show that adding a flow based module to the generative model p(x|z) increases both the likelihood and the fidelity of the samples. 
The experimental section seems sound with extensive results on CIFAR10 and some additional results on STL, LSUN, CelebA and ImageNet. 
 
Comments
Overall i find the paper well written and easy to understand and the experiments seems sound. My main criticism of the paper is the novelty of the proposed model. Adding a VAE and a GAN has been done before (e.g. Larsen 2016 as the authors cite as well). Outside of generative models the combination of MLE and GAN based losses have been studied in e.g. Super-Resolution (Shi et. al. 2016). In both papers of these papers the GAN based losses are added for exactly the same reasons as provided in this work i.e. to increase the sample fidelity. 
I’m unsure if adding a flow to the output of the VAE generative model have been done before, however in spirit the approach is quite similar in the ideas in “Variational Lossy Autoencoder” (Chen 2016) or PixelVAEs (Gulrajani 2016) where local covariance is added through conditional factorization.
 	 	 					
Questions
Q1) One criticism of the results is that the authors claim that the model achieves likelihood values competitive with state-of-the-art where the results seems to suggest that a more sound conclusion is that the the likelihoods are better than GANs but worse than MLE models such as PixelCNN++. Similarly for the Inception/FID scores where the model is better than VAEs but most of the time slightly worse than pure GAN models. ?
Q2) I find the claim “we propose a unified training approach that leverages coverage and quality based criteria” a bit overreaching. The loss in Eq(8) is simply the sum of a VAE and a GAN based loss and as such does not provide any unification of the two ?
Q3) Related to the previous question The KL-divergence interpretation of the GAN based loss assumes a bayes optimal discriminator. In practice this is usually never achieved why it is not really known what divergence GANs minimize if any (see e.g Fedus et. al. 2017 for a non-divergence minimization view). If the KL-divergenve minimization interpretation does not hold the proposed model is essentially a VAE with an auxiliary GAN loss biasing the model towards mode-seeking training? 
Q4) There haven’t been many successful GAN usages outside of of the CNN-realm which suggests that a successful GAN is tightly connected to the inductive bias provided by CNNs for images. Have the authors tried there model on something else than images? (I suspect that the GAN based L_Q loss will be hard to apply outside of the realm of images)
	
Overall i find the paper well written and easy to understand and the experiments seems sound.  However I think that closely related variants of the main contributions in the paper have been tried elsewhere which somewhat reduces the novelty of the proposed model. Given my comments above I think the paper is marginally below acceptance however I could be be convinced otherwise by e.g. a solid use-case for models like this?
Lastly, I'm sorry for the late review (I was called upon late to review the paper) - I hope that you will find the time for a proper rebuttal to my questions.