Normalizing Flows (NFs) are able to model complicated distributions p Y (y) with strong inter-dimensional correlations and high multimodality by transforming a simple base density p Z (z) through an invertible neural network under the change of variables formula. Such behavior is desirable in multivariate structured prediction tasks, where handcrafted per-pixel loss-based methods inadequately capture strong correlations between output dimensions. We present a study of conditional normalizing flows (CNFs), a class of NFs where the base density to output space mapping is conditioned on an input x, to model conditional densities p Y |X (y|x). CNFs are efficient in sampling and inference, they can be trained with a likelihood-based objective, and CNFs, being generative flows, do not suffer from mode collapse or training instabilities. We provide an effective method to train continuous CNFs for binary problems and in particular, we apply these CNFs to super-resolution and vessel segmentation tasks demonstrating competitive performance on standard benchmark datasets in terms of likelihood and conventional metrics.
Summary of the paper: 
The paper proposes an extension of Normalizing flows to conditional distributions. The paper is well written overall and easy to follow. Basically the conditional prior z|x = z=f_{\phi}(y,x), where x is the conditioning random variable, and we apply the change of variable formula to get the density of y|x . For example in super resolution y is the high res image and  x is the low res. image.  To sample from the models authors propose to use f^{-1}_{\phi}(z;x). 
The conditional modules are natural extensions of invertible blocks used in the literature (coupling layers, split priors, conditional coupling, 1x1 conv), where the conditioning is done on some hidden representations of the conditioning variable x (i.e one or multiple layers of NN).
Authors propose a dequantization for binary random variables (useful for segmentation applications), where they give an implicit model for the dequantizer (obtain a continuous variable from a discrete binary variable).
Author apply the method in two applications super-resolution and vessel segmentation. the method is compared to supervised learning of the corresponds between x and y and to others competitive methods in the literature and shows some advantage. 
Minor comments : 
- Formatting the bibliography is messed up and needs some cleaning , Figure 5 is also making formatting issues of the paper. 
- Figure 1 for sampling it should be f^-1_{\phi } and not f_{\phi}
Review: 
- Figure 2 is hard to get any idea of the sample quality would be good also to put the low resolution input to the algorithm . Also did you use a temperature sampling for the baseline ? otherwise the comparison is not fair.
- The Drive database is too small 20 training samples and 20 testing only? can the model be just overfitting?
- In the vessel implementation why do you drop the scaling modules? 
- The conditioning for the vessel implementation on x is on two layers , would be great to put all architectures of the models in details , and to show both sampling and training paths 
- It would be great to add the details of the skip connection used from the network processing x, and how ensure that the flow remains invertible.  
Overall this is a well written paper and a good addition to normalizing flows methods , some discussion of related works on conditional normalizing flows and more baselines with other competitive methods based on GANs for example would be helpful but not necessary. 
It would be great to add details of the architectures and on skip connections and how to ensure invertibility for this part in the model .