We propose a novel hierarchical generative model with a simple Markovian structure and a corresponding inference model. Both the generative and inference model are trained using the adversarial learning paradigm. We demonstrate that the hierarchical structure supports the learning of progressively more abstract representations as well as providing semantically meaningful reconstructions with different levels of fidelity. Furthermore, we show that minimizing the Jensen-Shanon divergence between the generative and inference network is enough to minimize the reconstruction error. The resulting semantically meaningful hierarchical latent structure discovery is exemplified on the CelebA dataset. There, we show that the features learned by our model in an unsupervised way outperform the best handcrafted features. Furthermore, the extracted features remain competitive when compared to several recent deep supervised approaches on an attribute prediction task on CelebA. Finally, we leverage the model's inference network to achieve state-ofthe-art performance on a semi-supervised variant of the MNIST digit classification task.
_________________________________________________________________________________________________________
I raise my rating on the condition that the authors will also address the minor concerns in the final version, please see details below.
_________________________________________________________________________________________________________
This paper proposes to perform Adversarially Learned Inference (ALI) in a layer-wise manner. The idea is interesting, and the authors did a good job to describe high-level idea, and demonstrate one advantage of hierarchy: providing different levels reconstructions. However, the advantage of better reconstruction could be better demonstrated.  Some major concerns should be clarified before publishing:
(1) How did the authors implement p(x|z) and q(z|x), or p(z_l | z_{l+1}) and q(z_{l+1} | z_l )? Please provide the details, as this is key to the reconstruction issues of ALI.
(2) Could the authors provide the pseudocode procedure of the proposed algorithm? In the current form of the writing, it is not clear what the HALI procedure is, whether (1) one discriminator is used to distinguish the concatenation of (x, z_1, ..., z_L), or (2) L discriminators are used to distinguish the concatenation of (z_l, z_{l+1}) at each layer, respectively?
The above two points are important. If not correctly constructed, it might reveal potential flaws of the proposed technique.
Since one of the major claims for HALI is to provide better reconstruction with higher fidelity than ALI. Could the authors provide quantitative results on MNIST and CIFAR to demonstrate this? The reconstruction issues have first been highlighted and theoretically analyzed in ALICE [*], and some remedy has been proposed to alleviate the issue.  Quantitative comparison on MNIST and CIFAR are also conducted. Could the authors report numbers to compare with them (ALI and ALICE)? 
The 3rd paragraph in Introduction should be adjusted to correctly clarify details of algorithms, and reflect up-to-date literature. "One interesting feature highlighted in the original ALI work (Dumoulin et al., 2016) is that ... never explicitly trained to perform reconstruction, this can nevertheless be easily done...". Note that ALI can only perform reconstruction when the deterministic mapping is used, while ALI itself adopted the stochastic mapping. Further, the deterministic mapping is the major difference of BiGAN from ALI. Therefore, more rigorous way to phrase is that "the original ALI work with deterministic mappings", or "BiGAN" never explicitly trained to perform reconstruction, this can nevertheless be easily done... This tiny difference between deterministic/stochastic mappings makes major difference for the quality of reconstruction, as theoretically analyzed and experimentally compared in ALICE. In ALICE, the authors confirmed further source of poor reconstructions of ALI in practice. It would be better to reflect the non-identifiability issues raised by ALICE in Introduction, rather than hiding it in Future Work as "Although recent work designed to improve the stability of training in ALI does show some promise (Chunyuan Li, 2017), more work is needed on this front."
Also, please fix the typo in reference as:
[*] Chunyuan Li, Hao Liu, Changyou Chen, Yunchen Pu, Liqun Chen, Ricardo Henao and Lawrence Carin. ALICE: Towards understanding adversarial learning for joint distribution matching. In Advances in Neural Information Processing Systems (NIPS), 2017.