Modelling highly multi-modal data is a challenging problem in machine learning. Most algorithms are based on maximizing the likelihood, which corresponds to the M(oment)-projection of the data distribution to the model distribution. The M-Projection forces the model to average over modes that can not be represented by the model. In contrast, the I(information)-Projection ignores such modes in the data and concentrates on the modes the model can represent. Such behavior is appealing whenever we deal with highly multi-modal data where it is more important to model single modes correctly instead of covering all the modes. Despite this advantage, the I-projection is rarely used in practise due to the lack of algorithms that can efficiently optimize it based on data. In this work, we present a new algorithm called Expected Information Maximization (EIM) for computing the I-projection solely based on samples for general latent variable models, where we focus on Gaussian mixtures models and Gaussian mixture of experts. Our approach applies a variational upper bound to the I-projection objective which decomposes the original objective into single objectives for each mixture component as well as for the coefficients, allowing an efficient optimization. Similar to GANs, our approach also employs discriminators but uses a more stable optimization procedures optimizing a tight upper bound. We show that our algorithm is much more effective in computing the I-projection than recent GAN approaches and we illustrate the effectiveness of our approach for modelling multi-modal behavior on two pedestrian and traffic prediction datasets.
The paper presents an algorithm to match two distributions with latent variables, named expected information maximization (EIM). Specifically, EIM is based on the I-Projection, which basically is equivalent to minimizing the reverse KL divergence (i.e. min KL[p_model || p_data]); to handle latent variables, an upper-bound is derived, which is the corresponding reverse KL divergence in the joint space. To minimize that joint reverse KL, a specific procedure is developed, leading to the presented EIM. EIM variants for different applications are discussed. Fancy robot-related experiments are used to evaluate the presented algorithm.
Overall, the paper is in good shape wrt the logic and the writing. My main concerns focus on the novelty (compared to existing methods that are similar but not discussed) and the experiments. For the former, reverse KL has been exploited before, both in the marginal space [1] and the joint one [2]. Other detailed comments are listed below.
As Eq 4 is for matching two joint distributions, discussions/comparisons should be made to reveal the novelty of the presented EIM over existing methods such as [2], etc.
In Figure 2 (b), the experimental settings for adversarial learning are not fair, as the discriminator is not fixed there. 
 
In Sec 4.4, it seems EIM is highly overlapped with VIPS. So what're the advantages of EIM here?
In Figure 3, how many steps for Generator and Discriminator are used for f-GAN? Does f-GAN finally converge? It would be helpful if some results are given to demonstrate the final state of each method.
In Eq. 9, adding the denominator q(z_i) will change the optimal solution. Why only add it to the first term?
In Section 5.3 and Figure 5, “SSD” might be a typo. 
[1] Adversarial Learning of a Sampler Based on an Unnormalized Distribution. AISTATS 2018.
[2] Symmetric Variational Autoencoder and Connections to Adversarial Learning. AISTATS 2019.