The application of multi-modal generative models by means of a Variational Auto Encoder (VAE) is an upcoming research topic for sensor fusion and bi-directional modality exchange. This contribution gives insights into the learned joint latent representation and shows that expressiveness and coherence are decisive properties for multi-modal datasets. Furthermore, we propose a multi-modal VAE derived from the full joint marginal log-likelihood that is able to learn the most meaningful representation for ambiguous observations. Since the properties of multi-modal sensor setups are essential for our approach but hardly available, we also propose a technique to generate correlated datasets from uni-modal ones.
This paper proposes an objective, M^2VAE, for multi-modal VAEs, which is supposed to learn a more meaningful latent space representation. To summarize my understanding of the proposed objective, in the bi-modal case, it combines both objectives of TELBO [1] and JMVAE-kl [2] with some hyperparameters to learn the uni-modal encoders. The terms of Eqns 7,8, and 9 are equivalent to TELBO and Eqns 9 and 10 are JMVAE-kl. It would be very beneficial for the readers if you could more clearly contrast your objective with the related work given how similar they are. 
Given these similarities between objectives, its unclear why JMVAE-Zero was chosen over JMVAE-kl as a baseline. Furthermore, the reasoning for the improvement of the ELBO of M^2VAE over the baselines in Section 5.3 is unclear, given the similarities between the objectives. 
The qualitative figures throughout the paper are hard to interpret. By looking at Fig 4., I cannot tell which latent space is best. 
“one can see from Fig. 4 that the most coherent latent space distribution was learned by the proposed M^2VAE” 
What is meant by ‘coherent latent space’? 
This paper was hard to follow and there are a number of typos throughout the paper. For instance, the labels within Fig 4 and the caption contradict themselves. If the clarity and quality of the writing could be improved then perhaps the contributions may become more evident.  
[1] R. Vedantam, I. Fischer, J. Huang, and K. Murphy. Generative Models of Visually GroundedImagination. ArXiv e-prints, May 2017.
[2] M. Suzuki, K. Nakayama, and Y. Matsuo. Improving Bi-directional Generation betweenDifferent Modalities with Variational Autoencoders. ArXiv e-prints, January 2018