We propose a new form of an autoencoding model which incorporates the best properties of variational autoencoders (VAE) and generative adversarial networks (GAN). It is known that GAN can produce very realistic samples while VAE does not suffer from mode collapsing problem. Our model optimizes λ-Jeffreys divergence between the model distribution and the true data distribution. We show that it takes the best properties of VAE and GAN objectives. It consists of two parts. One of these parts can be optimized by using the standard adversarial training, and the second one is the very objective of the VAE model. However, the straightforward way of substituting the VAE loss does not work well if we use an explicit likelihood such as Gaussian or Laplace which have limited flexibility in high dimensions and are unnatural for modelling images in the space of pixels. To tackle this problem we propose a novel approach to train the VAE model with an implicit likelihood by an adversarially trained discriminator. In an extensive set of experiments on CIFAR-10 and TinyImagent datasets, we show that our model achieves the state-of-the-art trade-off between generation and reconstruction quality and demonstrate how we can balance between mode-seeking and mass-covering behaviour of our model by adjusting the weight λ in our objective.
The paper proposes to replace the KL-divergence in VAE training with the lambda-jeffreys divergence of which the symmetric KL-divergence is a special case. The paper proposes a pure implicit likelihood approach that uses three discriminator models to estimate the KL-divergences. Experiments are conducted on CIFAR-10 and TinyImageNet and several scores are reported to show that the proposed method performs as good if not better than current approaches.
---------
I think the paper tries to achieve too much in too little space and foregoes scientific exactness for the sake of claiming SOTA. Since there is a difference between claiming SOTA on a task and validating a new method, the small amount of space makes it difficult to substantiate both claims at the same time. In the rest of the review i will try to substantiate the claim:
1. The paper claims on page 2: "These models do not have a sound theoretical justification about what distance [...] they optimize". While the paper tries to substantiate its claims by showing theoretically that it does the right thing using the optimal discriminator, it leaves the question open what happens with any other discriminator. The theory does not justify non-optimal solutions. It is argued on page 6 that non-optimality of the discriminator serves as some form of regularization, but  this requires some justification.
Moreover, the paper uses LPIPS to measure reconstruction quality - but this measure is a deep neural network. So if those measures are good enough to compare solutions with and the theoretical justification of the proposed method is shaky in practice - why not use LPIPS for training?
2. The paper proposes the discriminator in order to allow for an implicit likelihood. However, the r-function used in the experiments does not fulfill the property of a well defined likelihood, and Theorem 1 does not hold, since technically the KL-divergence is infinity. If we ignore this by adding a small amount of Gaussian noise around the sampled cyclical shifts - like the r' used in the experiments, we can easily write down the explicit likelihood function since:
r(y|x)=\sum_i w_i N(y|Shift_i(x), \sigma)
where Shift_i is the i-th shift in the set described in the paper and w_i its probability  p(y|q). So the explicit solution of theorem 1 can be written down and another ablation study would be training the method with the explicit formulation for this KL-term(i.e. only training two discriminator models). If the results are not equivalent, this implies that the discriminator does not reach the optimum. The implications of that should be discussed regarding 1. 
3.  Existing ablation studies are a bit of a straw-man: the paper compares changing r(y|x) by standard Gaussian or Laplace. However, we know that a large variance does not make any sense and almost all papers use tiny variances (e.g. in beta-VAE the beta-values tend to be very small, which is equivalent to small variances here). 
---------------------------
Smaller things
- Are the experimental results all with the same architecture for encoder/generator for all results you compared to? if not, the effect of that should also be tested.
- my personal biased view on the generated images is: it looks worse than alpha-GAN. Every reconstructed image has a grey tone and the generated images also offer a strong grey palette. The details don't look better as well.
- typo inroduce->introduce