Effective inference for a generative adversarial model remains an important and challenging problem. We propose a novel approach, Adversarial Inference by Matching priors and conditionals (AIM), which explicitly matches prior and conditional distributions in both data and code spaces, and puts a direct constraint on the dependency structure of the generative model. We derive an equivalent form of the prior and conditional matching objective that can be optimized efficiently without any parametric assumption on the data. We validate the effectiveness of AIM on the MNIST, CIFAR-10, and CelebA datasets by conducting quantitative and qualitative evaluations. Results demonstrate that AIM significantly improves both reconstruction and generation as compared to other adversarial inference models.
This paper presents a variant of the adversarial generative modeling
framework, allowing it to incorporate an inference mechanism. As such it is
very much in the same spirit as existing methods such as ALI/BiGAN. The
authors go through an information theoretic motivation but end up with the
standard GAN objective function plus a latent space (z) reconstruction
term. The z-space reconstruction is accomplished by first sampling z from
its standard normal prior and pushing that sample through the generator to
get sample in the data space (x), then x is propagated through an encoder
to get a new latent-space sample z'. Reconstruction is done to reduce the
error between z' and z.
Novelty: The space of adversarially trained latent variable models has
grown quite crowded in recent years. In light of the existing literature,
this paper's contribution can be seen as incremental, with relatively low novelty. 
In the end, the training paradigm is basically the same as InfoGAN, with
the difference being that, in the proposed model,  all the latent
variables are inferred (in InfoGAN, only a subset of the latent
variables are inferred) . This difference was a design decision on the part of the InfoGAN
authors and, in my opinion, does not represent a significantly novel
contribution on the part of this paper.  
Experiments: The experiments show that the proposed method is
better able to reconstruct examples than does ALI -- a result is not
necessarily surprising, but is interesting and worth further
investigation. I would like to understand better why it is that latent
variable (z) reconstruction gives rise to better x-space reconstruction.
I did not find the claims of better sample quality of AIM over ALI to be
well supported by the data. In this context, it is not entirely clear what
the significant difference in inception scores represents, though on this, the
results are consistent with those previously published
I really liked the experiment shown in Figure 4 (esp. 4b), it makes the
differences between AIM and ALI very clear. It shows that relative to ALI,
AIM sacrifices coherence between the "marginal" posterior (the distribution
of latent variables encoded from data samples) and the latent space
prior, in favor of superior reconstructions. AIM's choice of trade-off is
one that, in many contexts, one would happy to take as it ensures that
information about x is not lost -- as discussed elsewhere in the paper.
I view this aspect of the paper by far the most interesting. 
Summary,
Overall, the proposed AIM model is interesting and shows promise, but I'm
not sure how much impact it will have in light of the existing literature
in this area. Perhaps more ambitious applications would really show off the
power of the model and make it standout from the existing crowd.