Transportability from Multiple Environments with Limited Experiments: Completeness Results
This paper addresses the problem of mz-transportability, that is, transferring causal knowledge collected in several heterogeneous domains to a target domain in which only passive observations and limited experimental data can be collected. The paper first establishes a necessary and sufficient condition for deciding the feasibility of mz-transportability, i.e., whether causal effects in the target domain are estimable from the information available. It further proves that a previously established algorithm for computing transport formula is in fact complete, that is, failure of the algorithm implies non-existence of a transport formula. Finally, the paper shows that the do-calculus is complete for the mz-transportability class.
The authors address the problem (called mz-transportability) of inferring a causal relationship R in a target domain using causal knowledge obtained in multiple source domains with limited experiments.In the target domain only passive observations are available (or also some limited causal knowledge) which is not enough alone for the inference of R.Specifically the contributions are:1. A necessary and sufficient condition for deciding when causal effects in the target domain are estimable from the available information.2. A proof that a previously proposed algorithm (TR^mz) for computing the transport formula is complete.3. A proof that the do-calculus is complete for establishing mz-transportability of causal effects.*Quality & Clarity:This is mainly a theoretical paper and the claims are well-supported by the authors. It is well-organized and well-presented. However, the claims and proofs are often quite involved (as also pointed out by the authors) and I found it quite difficult to follow some parts (for example page 6) and fully understand the details. Nevertheless, I would probably attribute this more to the advanced theoretical analysis rather than to the lack of good presentation of the paper. I would propose for improvement:- Define what is "a maximal root set" in Definition 4 (like in [21]). - In Def.5: "(called an hedge)" -> "(called an hedge for P_x(y))".- The definition of an hedge after Definition 4 is confusing. You could include a more rigorous definition of what is an hedge there, instead of including it in Definition 5.- What do you mean by "separable"?- It would be beneficial for the reader to present some parts in a simpler way, if possible.*Originality & Significance:The paper builds on the NIPS 2013 paper ("[1] Transportability from Multiple Environments with Limited Experiments") which is adequately referenced. [1] included necessary OR sufficient conditions for transportability and proved only soundness of TR^mz. The current paper advances over [1] by providing conditions for complete characterization of transportability, proving that TR^mz is complete and proving that do-calculus is complete for the mz-transportability class. So, I consider the paper having a significant contribution. To be more explicit, you could add in the beginning of section 3 that the first part of this section (page 4) repeats from [1] or other publications (not only referring to them in page 4). Also, for eq. (2), add a reference to [1], where it is explain how eq. (2) is acquired.Some more comments:- Abstract, 3rd line: it seems that "limited experimental data" only refers to the target domain and not to the sources. Consider changing appropriately.- Motivation, 1st line: maybe missing a "that" after "experiments"? Consider rewriting sentence.- p.2, par.4: "the goal of....conditions the causal effect" -> "...conditions causal effects in the target domain"- First bullet before section 2: last line: "in the domains" -> "in the source domains"?- Section 2, line 2: you refer to the node S of Fig.1(a) which is missing: Consider adding the letter S in this square node of the figure.- Footnote 1: "..exclusively, on.." -> "..exclusively, of.."- Footnote 1: along with refs [13,14] you could also consider some of the following references:Ben-David et al. Analysis of representations for domain adaptationMansour et al. Domain adaptation: learning bounds and algorithmsMansour et al. Domain adaptation with multiple sourcesBen-David et al. A theory of learning from different domains- p.3, last line of 1st par: period before "our goal is"- explain what is V, U etc. in M_x- what is "IS" after Definition 2?- After Theorem 1: "It is not difficult to see..": add reference to [1]- middle p.5: period before "the result follows"- first line after Definition 4: remove "and" before "rooted"- p.7 middle: period before "refer to Appendix 2.."- Theorem 4:, last line: "contain" ->"contains", "spans"->"span", "a" ->"an" and "F'". What are sC-forests?- proof of Thm 4: don't you need capital C for c-forest?,   "shows" -> "showed"- After Thm 4 write that the proof is provided in the supp. material.- Maybe it would be better to use pi_c instead of pi_a for the source of Fig.1(c).- Supplementary, last page:   -- proof of Thm 5: remove one "the" (first line), remove one full stop (least line)  -- Proof of Corollary 1: "...from the previous Corollary": do you mean from the previous Theorem?