Modern generative models are usually designed to match target distributions directly in the data space, where the intrinsic dimensionality of data can be much lower than the ambient dimensionality. We argue that this discrepancy may contribute to the difficulties in training generative models. We therefore propose to map both the generated and target distributions to the latent space using the encoder of a standard autoencoder, and train the generator (or decoder) to match the target distribution in the latent space. The resulting method, perceptual generative autoencoder (PGA), is then incorporated with a maximum likelihood or variational autoencoder (VAE) objective to train the generative model. With maximum likelihood, PGAs generalize the idea of reversible generative models to unrestricted neural network architectures and arbitrary latent dimensionalities. When combined with VAEs, PGAs can generate sharper samples than vanilla VAEs. Compared to other autoencoder-based generative models using simple priors, PGAs achieve state-of-the-art FID scores on CIFAR-10 and CelebA.
The paper considers generative models and proposes to change the VAE approach in the following way. While VAEs assume a generative model (prior on the latent space + stochastic generator) and aim at learning its parameters so as to maximize the likelihood of the (training) data distribution, the authors propose to derive the learning objective from a different view. In my understanding, they consider the two composed mappings generator(encoder) and encoder(generator) and require the first one to have the data distribution as a fixpoint and the second one to have the latent prior as a fixpoint. Starting from this idea they derive an objective function for the case that the mappings are deterministic and then further enrich the objective by either likelihood based terms or VAE based terms. The new approach is analysed experimentally on MNIST, CIFAR and CelebA datasets. The authors report quantitative improvements in terms of Frechet inception distance (FID) as well as sharper samples (when compared to standard VAEs).
I find the main idea of the paper highly interesting and compelling. Nevertheless, I would not recommend to publish the paper in its present state . The technical part is in my view very hard to comprehend. This is partially due to the disadvantageous notation chosen by the authors. Furthermore, the derivation of the individual terms in the objective is hard to understand and the arguments given in the text are not convincing:
- The two terms in the objective related to the fixpoint of the encoder(generator) mapping seem to enforce a fixpoint that is a mixture of the latent Gaussian prior and the encoder image of the data distribution. It remains unclear to me, why this is a good choice.
- The derivation of the additional likelihood based terms and VAE based terms is in my view hard to understand.
It should be possible (and I believe, is possible) to derive a simpler objective ab initio, starting from the main idea of the authors. 
Side note: A close visual inspection of the presented generated samples seems to confirm the known doubts about the appropriateness of the FID measure for evaluation of generative models.