Multi-domain Causal Structure Learning in Linear Systems
We study the problem of causal structure learning in linear systems from observational data given in multiple domains, across which the causal coefficients and/or the distribution of the exogenous noises may vary. The main tool used in our approach is the principle that in a causally sufficient system, the causal modules, as well as their included parameters, change independently across domains. We first introduce our approach for finding causal direction in a system comprising two variables and propose efficient methods for identifying causal direction. Then we generalize our methods to causal structure learning in networks of variables. Most of previous work in structure learning from multi-domain data assume that certain types of invariance are held in causal modules across domains. Our approach unifies the idea in those works and generalizes to the case that there is no such invariance across the domains. Our proposed methods are generally capable of identifying causal direction from fewer than ten domains. When the invariance property holds, two domains are generally sufficient.
leveraged for the purpose of causal discovery, which is our focus in the paper. Moreover, it has been shown that causal modeling can be exploited to facilitate transfer learning [SJP + 12,ZSMW13]. This is because of the coupling relationship between causal modeling and distribution change: Causal models constrain how the data distribution may change, while distribution change exhibits such changes.
In this paper, we focus on linear systems, with observational data for the variables in the system given in multiple domains, across which the causal coefficients and/or the distribution of the exogenous noises may vary. We will present efficient approaches to exploiting such changes for causal structure learning. The proposed methods are based on the principle that although the cause and the effect variables are dependent, the mechanism that generates the cause variable changes independently from the mechanism that generates the effect variable across domains. The same principle was utilized in [ZHZ + 17] for exploiting non-stationary or heterogeneous data for causal structure learning. However, since that work considers a non-parametric approach, it is restricted to general independence tests among distributions, which may not have high efficiency. The remaining previous works on causal discovery from multiple domains usually assume that a certain type of invariance holds across domains [PBM16,GSKZ17]. Our approach unifies the idea in those works, and generalizes to the case that there is no such invariance across the domains. We present structure learning methods, which are generally capable of identifying causal direction from fewer than ten domains (as a special case, when the invariance property holds, two domains generally suffices).
The rest of the paper is organized as follows. We first introduce our approach for finding causal relations from multi-domain observational data in a system comprising two variables in Section 2. We propose our methods for identifying causal direction in Subsections 2.1 and 2.2. These methods are evaluated in Subsection 2.3. We generalize our method to causal structure learning in networks of variables in Section 3, where the generalized version of our methods are presented in Subsections 3.1 and 3.2 and evaluated in Subsection 3.4. We also present an approach for combining our proposed methods with standard conditional independence-based structure learning algorithms in Subsection 3.3. All the proofs are provided in the supplementary materials.
This paper proposes an approach to learn causal directionality -- first in the two-variable case, then generalized to DAGs, in the settings where multiple datasets of the same system are recorded across different "domains" (could be different time points, or different measurement instruments for example). The approach is based on an assumption called the "independent change condition" (ICC): for cause C and effect E, P(C|E) and P(E) should change independently across different domains. Further, linearity is assumed.  Under these assumptions, the authors develop interesting algorithms that first identify causal directions in the two-variable case, which is then generalized to structure learning. Importantly these algorithms could be used with a partial result from another structure learning method, such as PC, as input to identify further possible orientations.  Strong aspects of the paper are:  - The idea is innovative (the ICC concept itself is not new, but the algorithmic ideas appear to be) and leads to an interesting class of methods, which might be extensible further.  - The paper is well-written and clear (up to a few issues noted below).   - Due attention is given to the efficiency of the statistical testing procedures that underpin the algorithm. Two variants are presented in Section 3 that substantially increase power and reduce computation efforts compared to "naive" approaches. I emphasize this because structure learning papers sometimes tend to neglect the issues surrounding statistical testing.  Points of criticism are:   - Significance: I frankly speaking do not understand why we the ICC would be a good assumption. In the article, only two literature references are given (which by the way are hard to locate due to errors in the bibliography: DJM+10 appears to have a wrong author list and ZHZ+17 has a wrong title). I would have liked more motivation or perhaps a real-world example explaining why this would be a good assumption. If we allow all  parameters to change across domains, why would specific parameters change only independently?  - Clarity: I had trouble with Definition 3, which is central for understanding Theorem 4. What does "ordering" mean here precisely? It cannot be just a permutation of the variables, as one might suspect when looking at Theorem 4, because a permutation has no edges. From the examples given it seems that "ordering"  means instead "DAG" but that cannot be true either since otherwise all DAGs with no edges would be consistent with all other DAGs according to the definition. My working assumption was that order here means in fact a "connected partial order", in which case Theorem 4 appears to hold, but I might be misunderstanding this. It would be very useful if the authors could clarify this.  - Quality: The paper would benefit from a real-world example to demonstrate the applicability of the proposed techniques. Only simulated data is being used at the moment.  Minor points: - The paper should have a conclusion, even if it's only brief. - Assumption 1: I was wondering here why this is called "Faithfulness". You do explain this later after Assumption 2 (for the network case) and I'd suggest moving up this explanation. - Algorithm 1: I was wondering if it is really so important in which way you compute these regression coefficients and residual variances . Wouldn't this provide the same results as, for instance, simply running the regression of each variable on each other variable with higher order? I understand that you need to explain this somewhere to get to precise algorithm runtimes but lines 215 to 224 seemed to interrupt the flow of exposition a little and seemed to get in the way of your explanation of the actual structure learning approach. Perhaps it could just be mentioned that there's a way to compute all required parameters in time O(p^3) and put the details into supplement.  = After author feedback =  Thank you for your responses. A clarification of the "ordering" concept in the final version would indeed be greatly appreciated.