Identification and Estimation Of Causal Effects from Dependent Data
The assumption that data samples are independent and identically distributed (iid) is standard in many areas of statistics and machine learning. Nevertheless, in some settings, such as social networks, infectious disease modeling, and reasoning with spatial and temporal data, this assumption is false. An extensive literature exists on making causal inferences under the iid assumption [17,11,26,21], even when unobserved confounding bias may be present. But, as pointed out in [19], causal inference in non-iid contexts is challenging due to the presence of both unobserved confounding and data dependence. In this paper we develop a general theory describing when causal inferences are possible in such scenarios. We use segregated graphs [20], a generalization of latent projection mixed graphs [28], to represent causal models of this type and provide a complete algorithm for nonparametric identification in these models. We then demonstrate how statistical inference may be performed on causal parameters identified by this algorithm. In particular, we consider cases where only a single sample is available for parts of the model due to full interference, i.e., all units are pathwise dependent and neighbors' treatments affect each others' outcomes [24]. We apply these techniques to a synthetic data set which considers users sharing fake news articles given the structure of their social network, user activity levels, and baseline demographics and socioeconomic covariates.
The work presented in the paper is clearly of value.  The existing theory for identification and estimation of causal parameters in DAGs for IID data has been central to our understanding of causal inference, and developing analogous results for data under interference would be useful both to apply directly to data in which we know interference occurs and to better understand the potential impacts of violations of the IID assumption.  While the paper should be accepted, the current version could be substantially improved in both organization and in its discussion of several key issues, including generality, assumptions, temporal effects, and prior work.  Organization and Presentation  Some aspects of the organization make the paper challenging for readers.  Some sections do not provide a “roadmap” to the basic logic before plunging into the details, others do not present a high-level intuition for why a given theoretical result is being presented,  The entire paper would be substantially improved if the authors provided readers with a high-level roadmap to the overall reasoning of the paper, making clear the basic logic that allows an identification theory to be developed under interference (before plunging into the details of sections 2 and 3).  As I understand it, the outline of that logic is: 1) Represent models as latent variable chain graphs, 2) Divide the model into blocks, 3) Assume no interference between blocks, 4) Express identification theory by using truncated nested factorization of latent projection ADMGs.  This logic differs from the current abstract because it helps readers see how the paper achieves what it does, rather than just what the paper achieves.  The paper provides a great deal of theory with relatively little intuition to help readers understand the underlying concepts.  Such intuition would be relatively easy to provide.  For example, the fixing operation will be novel to most readers, and the paper would be improved by providing an intuition for what fixing corresponds to (if such an intution exists).  The intuition for conditioning, for example, is often presented in terms of partitioning of possible worlds.  If no such intuition exists, then it would be worth explaining up front that it is only a method for producing a nested factorization of an ADMG (or even relegating this to the supplemental material).  The abtract implies that unobserved confounding bias is a necessary (or at least likely) consequence of non-iid data, but the paper doesn’t make clear why this is so.  A quick description would be useful, so that readers would not have to consult outside papers, such as reference 14.  In contrast, Section 1 merely invokes latent confounding as a likely situation that appears in conjunction with interference.  More clarity on this point would help readers.  Generality  It is very difficult to assess the generality of the results presented in the paper.  Specifically, the details of the motivating example provided in Section 2 makes the results appear very narrow, but the more expansive langauge in the introduction and conclusion make the results seem much more general (“…complete identification theory, and estimation for dependent data.”).  Specifically, the work presented here applies to data consisting of a single object type and stable, symmetric relationships, such as friendship ties.  This is worth emphasizing in the introduction, because interference can occur in cases in which asymmetric relationships also occur (such as social network data, communications data, and citation data) and where multiple types of objects occur.  Assumptions  The method outlined in the paper appears to rely strongly on the presence of blocks (maximal sets of vertices in the model which each pair is connected by an undirected path) that lack interference.  This seems unlikely in many reasonable scenarios.  Interblock non-interference may be a common assumption, but that doesn’t mean that it is realistic.  The authors should either provide evidence that this is a plausible assumption in real applications or provide evidence that moderate violations of this assumption do not greatly affect the results.  Section 2 states that “a crucial assumption in our example is that…community membership does not directly influence sharing, except as mediated by social network activity of the user.”  This seems a moderately strong assumption, and it would be good to expand somewhat on both the likelihood of being able to gather data that meet such an assumption and the reasons why this assumption is made.  Temporal effects  The motivating example (and other plausible examples) seems extremely difficult to analyze without explicit consideration of the time-series of behavior.  This is not just a simple statement that temporal information aids causal identification.  Instead, the point here is that the system described almost certainly has substantial feedback effects and time-varying exogenous influences, and it may never reach any form of equilibrium.  Thus, it is unclear what a data sample corresponds to here.  The paper would be improved greatly by describing in more detail the underlying assumptions of data collection.  Prior work  The paper ignores a moderately large literature on inferring causal models of non-iid data that has appeared in the graphical models literature over the past decade.  Relevant papers include:  Marc Maier (2013). Reasoning about independence in probabilistic models of relational data. arXiv:1302.4381.  Lee & Honavar (2016). On learning causal models from relational data. AAAI 2016.  Maier et al. (2013). A sound and complete algorithm for learning causal models from relational data.  UAI 2013.  Arbour et al. (2016). Inferring network effects from observational data. KDD 2016.  These papers do not subsume or contradict the current paper, but the current paper would be improved by contrasting the assumptions of these papers with the approach taken here.  Summarize the strengths and weaknesses of the submission, focusing on each of the following four criteria: quality, clarity, originality, and significance.