Causal Inference on Time Series using Restricted Structural Equation Models
Causal inference uses observational data to infer the causal structure of the data generating system. We study a class of restricted Structural Equation Models for time series that we call Time Series Models with Independent Noise (TiMINo). These models require independent residual time series, whereas traditional methods like Granger causality exploit the variance of residuals. This work contains two main contributions: (1) Theoretical: By restricting the model class (e.g. to additive noise) we provide general identifiability results. They cover lagged and instantaneous effects that can be nonlinear and unfaithful, and non-instantaneous feedbacks between the time series. (2) Practical: If there are no feedback loops between time series, we propose an algorithm based on non-linear independence tests of time series. We show empirically that when the data are causally insufficient or the model is misspecified, the method avoids incorrect answers. We extend the theoretical and the algorithmic part to situations in which the time series have been measured with different time delays. TiMINo is applied to artificial and real data and code is provided.
I read the author feedbacks: - Theorem 1 (i) 
      (our main result) does not require acyclicity of the summary time graph. 
      Note that acyclicity is only one of two sufficient conditions. I 
      found that I did not reflect enough that the acyclic assumption is one of 
      two sufficient conditions. Thanks. I changed the score up. I still 
      don't understand why they tested their method only on data which are 
      generated under the acyclic assumption on the summary time graph. That 
      seems to be more limited cases. Summary of the 
      paper: This paper considers a class of structural equation models for 
      times series data. The models allow nonlinear instantaneous effects 
      and lagged effects. On the other hand, Granger-causality based methods do 
      not allow instantaneous effects and a linear non-Gaussian method TS-LiNGAM 
      (Hyvarinen et al., ICML2008, JMLR2010) assumes linear effects. 
      Then they gave Theorem 1, which shows the identifiability 
      conditions of the model. The conditions are i) the functions that causally 
      relates variables come from a class of functions called IFMOC considered 
      in Peters et al. [2011b], ii) the faithfulness and iii) the summary time 
      graph is acyclic (They define the summary time graph is a graph that 
      contains all the nodes of the time series and an arrow X^i (X at time 
      point i) and X^j, i ¥neq j, if there is an arrow from X^i_{t-k} to X_t^i 
      in the full time graph for SOME k in lines 53-55). The theorem 1 is an 
      extension of Peters et al. [2011b] to a time series case and a partial 
      extension of Hyvarinen et al. (ICML2008, JMLR2010) to nonlinear cases. 
      Hyvarinen et al. proves the identifiability of their model assuming 
      linearity and non-Gaussian residuals (disturbances or external 
      influences), in which the summary graph needs not to be acyclic. 
      They further presented an estimation algorithm (Algorithm 1), 
      which is a natural extension of Mooij et al. [2009] to time series cases. 
      They use independence test to detect possible violations of model 
      assumptions. Their algorithm can be applied (with small modifications) to 
      shifted time series cases, which are often observed in fMRI data. 
      They conducted artificial data experiments to test their 
      algorithms under various settings including latent confounders with time 
      lag but still assumed that the summary graph is acyclic. They compared 
      their results with a Granger-causality-based linear method and a linear 
      non-Gaussian method TS-LiNGAM (Hyvarinen et al., ICML2008). Their methods 
      are better in some nonlinear cases. Finally, they tested some real 
      datasets. Their methods worked better in some cases and were comparable to 
      linear non-Gaussian TS-LiNGAM in some cases. Pros.: - It is 
      interesting to extend TS-LiNGAM (Hyvarinen et al., ICML2008; JMLR2010) to 
      nonlinear cases. - It is practically important to consider shifted 
      time series cases. Cons.: - The assumption in Theorem 1 that 
      the summary graph is acyclic seems too restrictive unless I miss something 
      important. It means that it does not happen that X^1_t-1 causes X^2_t and 
      X^2_t-1 causes X^1_t, doesn’t it?? They claim that their methods is a 
      kind of safe methods since it outputs `I don’t know’ if independence 
      between exploratory variables and residuals is statistically rejected, 
      which implies some violations of the model assumptions. The idea is not 
      very new. It can be found in many papers, e.g., Entner and Hoyer 
      (AMBN2010), Tashiro et al. (ICANN2012), Shimizu and Kano (2008, JSPI) and 
      Hoyer et al. (NIPS2009). Suggestions: Would it be possible to 
      prove Theorem 1 without assuming that the summary graph is acyclic? 
      - They claim that their method is a kind of safe methods since it 
      outputs `I don’t know’ if independence between exploratory variables and 
      residuals is rejected. The idea is not very new. For example, they should 
      at least prove that explanatory variables and residuals are not 
      independent if latent confounders exist to make their contribution 
      clearer. Entner and Hoyer (AMBN2010) and Tashiro et al. (ICANN2012) proved 
      that in linear non-Gaussian cases. Comments An extension 
      of TS-LiNGAM to cases with latent confounders with time lag might be 
      Kawahara, S. Shimizu, and T. Washio. Analyzing relationships among 
      ARMA processes based on non-Gaussianity of external 
      influences.Neurocomputing, 74(12-13): 2212-2221, 2011. It considers 
      ARMA instead of AR model to model temporal dependency. Another extension 
      of TS-LiNGAM to cases with instantaneous confounders would be Gao and 
      H. Yang. Identifying structural VAR model with latent variables using 
      overcomplete ICA. Far East Journal of Theoretical Statistics, 40(1): 
      31-44, 2012. In the last year NIPS, TS-LiNGAM was extended to cases 
      with variance-dependent external influences. Z. Chen, K. Zhang, and L. 
      Chan. Causal discovery with scale-mixture model for spatiotemporal 
      variance dependencies. In Advances in Neural Information Processing 
      Systems 25 (NIPS2012), pp. xx-xx, 2012. Taking these developments in 
      linear non-Gaussian models into account, the proposed nonlinear model 
      assuming the sample time graph is acyclic might be a small contribution, 
      although extending to nonlinear models certainly is an interesting topic. 
      In Experiment 11, they say that “TiMINo does not decide since the 
      model leads to dependent residuals. We nevertheless analyzed ...” I didn’t 
      follow why they decided to continue to analyze the data even if the 
      residuals were found to be dependent. - They say that “we now 
      refer to entries larger than 0.5 as causation”. I didn’t follow why they 
      choose the threshold 0.5.