Perceptual Generative Autoencoders
Modern generative models are usually designed to match target distributions directly in the data space, where the intrinsic dimensionality of data can be much lower than the ambient dimensionality. We argue that this discrepancy may contribute to the difficulties in training generative models. We therefore propose to map both the generated and target distributions to the latent space using the encoder of a standard autoencoder, and train the generator (or decoder) to match the target distribution in the latent space. The resulting method, perceptual generative autoencoder (PGA), is then incorporated with a maximum likelihood or variational autoencoder (VAE) objective to train the generative model. With maximum likelihood, PGAs generalize the idea of reversible generative models to unrestricted neural network architectures and arbitrary latent dimensionalities. When combined with VAEs, PGAs can generate sharper samples than vanilla VAEs. Compared to other autoencoder-based generative models using simple priors, PGAs achieve state-of-the-art FID scores on CIFAR-10 and CelebA.
The main contribution of the paper is in the novel training procedure in which the target distribution and the generated distributions are mapped to a latent space where their divergence is minimised. Training procedures for the same based on MLE and VAE are presented as well. The idea is quite interesting but there are certain problems in clarity regarding the loss terms and the precise need for them. It appears that the work is proposing a novel framework for training VAEs where instead of comparing the target and generated distribution in the data space (the standard ), the loss is minimised by minimizing the divergence between these distributions after mapping them onto a latent space. To achieve this objective the author’s design two loss terms in addition to the standard reconstruction error. Authors provided two approaches for their training method based on MLE and VAE. Their framework is then experimentally evaluated on the standard dataset and demonstrate superior results; better-generated images and lower FID score) and provide ablation results. The explanation of the Equation 2 is confusing, where the Figure-1a and the equation seem to agree but the subsequent description says that should map to but Equation-2 will bring closer to . Is there something missing here? The notation and the derivation of the MLE approach in Section 3.3 are not clear. This section would need rewriting with clearer and compact notation for better readability. The main idea is quite clear but how this training procedure achieves that is not coming out clearly in this version. Thus I feel the paper, though has good content, is not publishable in the current format.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Perceptual Generative Autoencoders
Modern generative models are usually designed to match target distributions directly in the data space, where the intrinsic dimensionality of data can be much lower than the ambient dimensionality. We argue that this discrepancy may contribute to the difficulties in training generative models. We therefore propose to map both the generated and target distributions to the latent space using the encoder of a standard autoencoder, and train the generator (or decoder) to match the target distribution in the latent space. The resulting method, perceptual generative autoencoder (PGA), is then incorporated with a maximum likelihood or variational autoencoder (VAE) objective to train the generative model. With maximum likelihood, PGAs generalize the idea of reversible generative models to unrestricted neural network architectures and arbitrary latent dimensionalities. When combined with VAEs, PGAs can generate sharper samples than vanilla VAEs. Compared to other autoencoder-based generative models using simple priors, PGAs achieve state-of-the-art FID scores on CIFAR-10 and CelebA.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The main contribution of the paper is in the novel training procedure in which the target distribution and the generated distributions are mapped to a latent space where their divergence is minimised. Training procedures for the same based on MLE and VAE are presented as well. The idea is quite interesting but there are certain problems in clarity regarding the loss terms and the precise need for them. It appears that the work is proposing a novel framework for training VAEs where instead of comparing the target and generated distribution in the data space (the standard ), the loss is minimised by minimizing the divergence between these distributions after mapping them onto a latent space. To achieve this objective the author’s design two loss terms in addition to the standard reconstruction error. Authors provided two approaches for their training method based on MLE and VAE. Their framework is then experimentally evaluated on the standard dataset and demonstrate superior results; better-generated images and lower FID score) and provide ablation results. The explanation of the Equation 2 is confusing, where the Figure-1a and the equation seem to agree but the subsequent description says that should map to but Equation-2 will bring closer to . Is there something missing here? The notation and the derivation of the MLE approach in Section 3.3 are not clear. This section would need rewriting with clearer and compact notation for better readability. The main idea is quite clear but how this training procedure achieves that is not coming out clearly in this version. Thus I feel the paper, though has good content, is not publishable in the current format.
assistant
The main contribution of the paper is in the novel training procedure in which the target distribution and the generated distributions are mapped to a latent space where their divergence is minimised. Training procedures for the same based on MLE and VAE are presented as well. The idea is quite interesting but there are certain problems in clarity regarding the loss terms and the precise need for them. It appears that the work is proposing a novel framework for training VAEs where instead of comparing the target and generated distribution in the data space (the standard ), the loss is minimised by minimizing the divergence between these distributions after mapping them onto a latent space. To achieve this objective the author’s design two loss terms in addition to the standard reconstruction error. Authors provided two approaches for their training method based on MLE and VAE. Their framework is then experimentally evaluated on the standard dataset and demonstrate superior results; better-generated images and lower FID score) and provide ablation results. The explanation of the Equation 2 is confusing, where the Figure-1a and the equation seem to agree but the subsequent description says that should map to but Equation-2 will bring closer to . Is there something missing here? The notation and the derivation of the MLE approach in Section 3.3 are not clear. This section would need rewriting with clearer and compact notation for better readability. The main idea is quite clear but how this training procedure achieves that is not coming out clearly in this version. Thus I feel the paper, though has good content, is not publishable in the current format.