We provide a critical view of the univariate Structural Causal Model. We show that by comparing the individual complexities of univariate cause and effect, in the Structural Causal Model, one can identify the cause and the effect, without considering their interaction at all. The entropy of each variable is ineffective in measuring the complexity, and We propose We suggest to capture it the complexity by the reconstruction error of an autoencoder that operates on the list of sorted samples the percentiles of the distribution. Comparing the reconstruction errors of the two autoencoders, one for each variable, is shown to perform well on the accepted benchmarks of the field. Hence, the decision as to which of the two is the cause and which is the effect may not be based on causality but on complexity. In the multivariate case, where one can ensure that the complexities of the cause and effect are balanced, we propose a new adversarial training method that mimics the disentangled structure of the causal model. We extend the results of Zhang & Hyvrinen (2010) to the multidimensional case, showing that such modeling is only likely in the direction of causality. Furthermore, the learned model is shown theoretically to perform the separation to the causal component and to the residual (noise) component. Our multidimensional method obtains a significantly higher accuracy than the literature methods.
Edit after author rebuttal and author additions:
I have updated my score from a weak reject (3) to a weak accept (6).
Justification:
1. The authors have pointed out that I misunderstood one of their contributions: pointing out that they are demonstrating that the univariate case is an insufficient setting to test causal discovery methods because it can be done without even looking at the conditional distributions (just the marginals). This contribution seems important to orient future work. They have made this more clear in their recent upload and would likely make it even more clear in a camera-ready version.
2. The authors have made their contribution to the multivariate setting more substantial by adding evaluation on the MOUS-MEG real-world dataset and have better positioned their work relative to others by adding comparisions to multivariate extensions of PNL and CGNN.
====================================================================================================
Original Review:
Summary: The authors focus on the problem of inferring whether the causal structure X —> Y or Y —> X. They first consider the case where both X and Y are scalar (univariate) random variables and then consider the case where X and Y are vector-valued (multivariate) random variables. In the scalar case, motivated by the idea that the effect could be less entropic than the cause (due to data processing inequality), they introduce a method based on comparing reconstruction losses of X and Y and show competitive results in Tables 1 and 2. They establish that this method is not sufficient for the multivariate case in Lemma 2 and move to a new method for the multivariate case. They prove identifiability for this new method in for the multivariate case in Section 4.2 and claim state-of-the-art (SOTA) results in Table 3. 
Main contributions:
- Presents a causal discovery technique for the univariate cases that only examines the marginal distributions of X and Y and seems fairly competitive (Tables 1 and 2)
- Extends the post-nonlinear identifiability analysis of Zhang & Hyv¨arinen (2009) from scalars to vectors and proved that their method will actually identify the correct causal direction
- Demonstrates competitive experimental results for both their univariate method
- Claims SOTA results for their multivariate method
Decision: I lean toward rejecting this paper because 1) I have several questions about the univariate case (see below) that would need to be resolved before I lean toward accept, 2) although I am not too familiar with the literature, I believe that this paper may be missing key related work that also uses independence testing for causal discovery (see, e.g., Heinze-Deml et al. (2017)’s Invariant Causal Prediction for Nonlinear Models), and 3) I am not yet convinced that the comparison done in Table 3 is fair and exhaustive.  
Sufficient reason to accept: If the theorems in Section 4.2 are found to checkout, and the SOTA results in Table 3 are found to be fair, exhaustive comparisons to the previous SOTA, their contribution to the multivariate case would seem to be sufficient for acceptance. I believe more discussion between the authors and reviewers is necessary here.
 Questions about univariate case:
 1. The motivation for the first method (entropy decreasing along a Markov chain due the data processing inequality) seems to only be valid when Y := f(X), but not necessarily when Y := f(X) + E. For example, let f be the identity function and E be independent to X. How did you resolve this argument against the intuition? 
2. Also, I thought the data processing inequality relates mutual information between variables, not necessarily their entropies. Can you make this connection more clear? 
Context for questions 3 and 4: In Section 3.1, you write, “estimating the entropy of each random variable from its samples does not present a consistent difference between the entropies h(X)  and h(Y). Our method, therefore, computes an alternative complexity score for X and, independently, for Y.” You then go on to link the entropy to the reconstruction error (your method) in Lemma 1 and show competitive results in Tables 1 and 2.
3. Why do you want to link the reconstruction error to entropy if you found a purely entropy-based method did not work?
4. Why did the purely entropy-based method not work while your method worked if the two are linked?
Questions about multivariate case:
5. Are you certain that BivariateFit and ANM are the only models that you should be comparing against for this multivariate setting?
6. What is CGNN’s runtime? Would you be able to compare against CGNN in time for a potential camera-ready version of this paper?