Learned joint representations of images and text form the backbone of several important cross-domain tasks such as image captioning. Prior work mostly maps both domains into a common latent representation in a purely supervised fashion. This is rather restrictive, however, as the two domains follow distinct generative processes. Therefore, we propose a novel semi-supervised framework, which models shared information between domains and domain-specific information separately. The information shared between the domains is aligned with an invertible neural network. Our model integrates normalising flow-based priors for the domain-specific information, which allows us to learn diverse many-to-many mappings between the two domains. We demonstrate the effectiveness of our model on diverse tasks, including image captioning and text-to-image synthesis.
Summary:
The paper proposes a model for joint image-text representations 
The paper proposes a model fro cross-domain generative tasks, specifically image captioning and text-to-image synthesis. The proposed Latent Normalizing Flows for Many-to-Many Mappings uses normalizing flows to model complex joint distributions. The latent representation consist of domain-specific representation and cross-domain information shared across image and text using invertible metrics. 
Novelty: 
- The paper explores an interesting area of learning representations for cross-domain tasks such as image captioning and text-to-image synthesis.
- The model is well explained.  Section 3 explains the model formulation nicely, has consistent notation and slowly  builds up to the final formulation by explaining each component concisely.
- Recent methods like VQ-VAE [1]  have shown promising results for image generation. The related work doesn't provide any discussion regarding that. 
Experiments / Analysis:
- The model contains exhaustive experiments for both image-captioning and image generation. The model performs on-par or beat state of the art methods on both perceptual and diversity metrics. On diversity metrics, the model performs much better than other recent methods like Seq-CVAE (which arrived on ArXiv only a few weeks prior to the submission deadline) and POS.
- The model also shows results on text-to-image synthesis comparing with multiple baselines and various diversity metrics and inception score.
- While the proposed methods beats existing diversity and perceptual metrics, it'd be good to also run a human study since these metrics are only a proxy to human judgement.
- Apart from showing empirical result, the paper can benefit from providing Insights what the domain specific representation has learnt and what the cross-domain representation learnt.
- Can the model benefit from training on unaligned image and textual data to learn better domain specific representations? 
Clarity: 
- Ablations not clear: It's not a 100% clear from the paper what LNFMM-MSE and LNFMM-TXT mean. "LNFMM-TXT contains unsupervised dimensions only for the text distribution and
all encoded image features are used for supervision, i.e.without fφv" What does this sentence mean?  Similarly, it's not clear what "LNFMM (semi-supervised, 30% labeled)" mean?
- It's also not clear why the authors call the approach a semi supervised setup? For instance,  the paper relies on supervision from  paired image-caption data to train the model. 
[1] Neural Discrete Representation Learning; Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu
**Update after rebuttal**
Thank you for clarifications to my questions regarding the ablations and  using unaligned image and textual data to learn better domain specific representations. The visualizations in Appendix A.4 are also somewhat helpful in understanding what the representations have learnt. After reading the rebuttal, and considering that they will run the human study to further validate their approach in the final manuscript, I am happy to raise the score.