Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
An important goal common to domain adaptation and causal inference is to make accurate predictions when the distributions for the source (or training) domain(s) and target (or test) domain(s) differ. In many cases, these different distributions can be modeled as different contexts of a single underlying system, in which each distribution corresponds to a different perturbation of the system, or in causal terms, an intervention. We focus on a class of such causal domain adaptation problems, where data for one or more source domains are given, and the task is to predict the distribution of a certain target variable from measurements of other variables in one or more target domains. We propose an approach for solving these problems that exploits causal inference and does not rely on prior knowledge of the causal graph, the type of interventions or the intervention targets. We demonstrate our approach by evaluating a possible implementation on simulated and real world data.
Summary: this paper proposes a new approach to domain adaptation that relies on the identification of a separating feature set, conditional on which the distribution of a variable of interest is invariant under a certain intervention. The problem that the authors set out to solve is a vey ambitious and difficult one. I think that they are partially successful in their attempt, although I have a number of serious reservations. Detailed comments follow:   1) I found the paper generally very well written and clear. I come from a `potential outcomes’ school of causal inference, and my knowledge of DAGs is not extensive — even so, the authors made a reasonable effort to introduce all needed concepts and I was able to follow the details of the exposition. I believe this article to be accessible to most researchers who have a solid background in causal inference, even if their knowledge of DAGs is limited.  2) I appreciate the fact that the authors (mostly) avoided making grandiose claims about the reach of their method. They acknowledge some of its limitations openly. I also greatly appreciate the innate conservativeness of their algorithm (if we accept their assumption) which may refuse to make predictions if it cannot find a separating feature set.  3) If we accept the assumptions made by the authors, then although the basic idea is not groundbreaking (the authors acknowledge as much in footnote 4 on page 5), their contribution to identifying separating sets of features is original and interesting.  3) My first serious criticism is with the assumptions being made. First of all, I think that more space should have been devoted to discussing the assumptions. I understand that 10 pages doesn’t leave a lot of space, but the entire premise of this work rests on these assumptions. The Markov and Faithfulness assumptions being standard, it’s ok to not expand on them too much (although it never hurts to discuss an assumption). Assumptions 1 and 2 are another matter entirely — neither is standard and both are crucial to the rest of the paper. While I am roughly ok with Assumption 1, I have serious reservations about Assumption 2. Specifically, Assumption 2(ii) and Assumption 2(iii) seem to be assuming away most of the problem. At a minimum, the authors should provide (more than one) concrete example where both assumptions are credible. My main contention is that 2(ii) and 2(iii) assume a strong prior knowledge of the causal mechanisms in the system, and I cannot envision a scenario in which an individual with such knowledge would still have a need for domain adaptation.  4) My second serious criticism is with the practical aspect of this method. The authors acknowledge the computational complexity of the approach, but don’t discuss the details of this limitation. Concretely, how many variables can this method handle? Both examples in Section 3 deal with very few variables, far fewer than what would be required in most practical scenarios I can think of. The authors state in Section 4 that “more work remains to be done on the implementation side […] scaling up to more variables”. Do you have a sense for what the algorithmic complexity of the problem is? How likely is it that any significant scaling improvements (say handling ~50 variables) will occur soon? Do you have concrete ideas about how to get there?   Additional minor comments:  1) The first sentence of Example 1 is awkward. It’s a subordinate clause that isn’t attached to a main clause. 2) Same comment for the first sentence of Task 1. 3) I found Example 1 to be unclear. You should make it clear earlier that the example assumes $P(Y | X_1, C_1 = 0) = P(Y | X_1, C_1=1)$ instead of writing it at the end. 4) The labels of the plots in Figure 3 are way too small.