A central question of representation learning asks under which conditions it is possible to reconstruct the true latent variables of an arbitrarily complex generative process. Recent breakthrough work by Khemakhem et al. (2019)  on nonlinear ICA has answered this question for a broad class of conditional generative processes. We extend this important result in a direction relevant for application to real-world data. First, we generalize the theory to the case of unknown intrinsic problem dimension and prove that in some special (but not very restrictive) cases, informative latent variables will be automatically separated from noise by an estimating model. Furthermore, the recovered informative latent variables will be in one-to-one correspondence with the true latent variables of the generating process, up to a trivial component-wise transformation. Second, we introduce a modification of the RealNVP invertible neural network architecture (Dinh et al., 2016) which is particularly suitable for this type of problem: the General Incompressibleflow Network (GIN). Experiments on artificial data and EMNIST demonstrate that theoretical predictions are indeed verified in practice. In particular, we provide a detailed set of exactly 22 informative latent variables extracted from EMNIST.
This paper extends recent work by Khemakhem et al on nonlinear ICA to allow for unknown number of generative factors. This is tackling an important problem in the field of generative modeling, where one would like to extract the generative factors of a dataset that independently control its features (i.e. disentanglement).
The paper is very clearly written, does a great job at motivating the problem and presenting the recent results from Khemakhem et al, before extending them with some simple theorems and demonstrating their application on toy data + EMNIST.
I think that this research direction is extremely promising, and obtaining a theoretical understanding of when/why disentangling could work would be particularly valuable to the field, and I would lean towards acceptance so that this work gets more attention.
It is slightly surprising however that their empirical results seem to indicate that these theoretical conditions do not seem to be necessary, which should be investigated further (and might help illustrate the dichotomy in some claims and results obtained in the disentangling literature recently).
1.	It was unclear to me how one should/would decide what to use for $u$ or what to leave out to be factorized by the method. 
	This may be out of scope for this current work, but one way to answer that would be to leverage datasets with more fully labelled factorised data (e.g. dSprites [1]) and present how one should leverage these with $u$, in order to identify the original generative factors.
2.	Related, using EMNIST was interesting, but given the lack of “accepted” generative factors to be recovered, it is hard to tell if the 22 variables found are “correct” or more similar to using a generative model which would entangled the data more (and hence would falsely introduce extra latent variables).
3.	The toy dataset, with its random RealNVP network to produce the data, was less “mixed” than I expected, looking at Figure 2B. In its projected view, the clusters are still rather easy to identify by eye, which surprised me somehow? Could you comment a bit more on how difficult is the task, or present a VAE baseline that would fail to explain that data?
4.	I did not see how the threshold for selecting 22 latent variables on EMNIST was set? Was the 23rd latent variable significantly less informative? The spectrum on Figure 3A was not precise enough to assess this fairly and the single mention of “measured by the standard deviations of test data transformed to the latent space” was too vague to reverse-engineer the decision.
5.	It was interesting to read about the observations of when this method should fail. It would be interesting if a dataset with explicit “gaps” would be constructed to analyze this case.
6.	Figure 4 might benefit from mentioning “what” each variable controls for directly in their individual caption / on top of them, instead of having to read this through in the full figure caption.
7.	The current model in the end is modeling the latent state using a mixture of gaussians (although these now have a theoretical connection to the true generative factors). How much does this differ to existing generative models using VAEs with a mixture of gaussian prior [2, 3]?
References:
[1] dSprites dataset: https://github.com/deepmind/dsprites-dataset 
[2] https://arxiv.org/abs/1611.02648
[3] https://arxiv.org/abs/1902.03717