Learning causal relations from observational time series with nonlinear interactions and complex causal structures is a key component of human intelligence, and has a wide range of applications. Although neural nets have demonstrated their effectiveness in a variety of fields, their application in learning causal relations has been scarce. This is due to both a lack of theoretical results connecting risk minimization and causality (enabling function approximators like neural nets to apply), and a lack of scalability in prior causal measures to allow for expressive function approximators like neural nets to apply. In this work, we propose a novel causal measure and algorithm using risk minimization to infer causal relations from time series. We demonstrate the effectiveness and scalability of our algorithms to learn nonlinear causal models in synthetic datasets as comparing to other methods, and its effectiveness in inferring causal relations in a video game environment and real-world heart-rate vs. breath-rate and rat brain EEG datasets.
This paper aims to estimate time-delayed, nonlinear causal influences from time series under the causal sufficiency assumption. It is easy to follow and contains a lot of empirical results. Thanks for the results, but I have several questions.
First, In Theorem 2, which seems to be a main result of the paper, the authors were concerned with the condition when W_{ji} >0, but there is not conclusion if W_{ji} =0. In order to correctly estimate causal relations from data, both cases must be considered.
Second, the conclusion of Theorem 2 seems to be flawed. Let me try to make it clear with the following example. Suppose x^1_{t-2} directly causes x^2_{t-1} and that x^2_{t-1} directly causes x^3_{t}, without a direct influence from x^1_{t-2}  to x^3_{t}. Then when minimizing (2), we have the following results step by step:
1) The noise standard deviation in x^2_{t-1}, denoted by \eta_2, may be non-zero. This is because we minimize a tradeoff of the prediction error (the first term in (2)) and a function of the reciprocal of the noise standard deviation \eta_2 (the second term in (2)), not only the prediction error.
2) If \eta_2 is non-zero, then x^1_{t-2} will be useful for the purpose of predicting x^3_{t}. (Note that if \eta_2 is zero, then x^1_{t-2} is not useful for predicting x^3_{t).) From the d-separation perspective, this is because x^1_{t-2} and x^3_{t} are not d-separated by x^2_{t-1} + \eta_2 \cdot \epsilon_2, although they are d-separated by x^2_{t-1}. Then the causal Markov condition tells use that x^1_{t-2} and x^3_{t} are not independent conditional on x^2_{t-1} + \eta_2 \cdot \epsilon_2, which means that x^1_{t-2} is useful for predicting x^3_{t}.
3) Given that x^1_{t-2} is useful for predicting x^3_{t}, when (2) is minimized, \eta_1 will not go to infinity, resulting in a non-zero W_{13), which *mistakenly* tells us that X^{1}_{t-1} directly structurally causes x^{(3)}_t.
This illustrates that the conclusion of Theorem 2 may be wrong.  I believe this is because the proof of Theorem 2 is flawed in lines 5-6 on Page 16. It does not seem sensible to drop X^{j}_{t-1} + \eta_X \cdot \epsilon_X and attain a smaller value of the cost function at the same time. Please carefully check it, especially the argument given in lines 10-13.
Third, it is rather surprising that the authors didn't mention anything about the traditional causal discovery methods based on conditional independence relations in the data, known as constraint-based methods, such as the PC algorithm (Spirtes et al., 1993), IC algorithm (Pearl, 2000), and FCI (Spirtes et al., 1993). Such methods are directly applicable to time-delayed causal relations by further considering the constraint that effects temporally follow the causes. 
Fourth, please make it clear that the proposed method aims to estimate "causality-in-mean" because of the formulation in terms of regression. For instance, if x^j_{t-1} influences only the variance of x^i_{t}, but not its mean, then the proposed method may not detect such a causal influence, although the constraint-based methods can.
Any response would be highly appreciated.