Avoiding Discrimination through Causal Reasoning
Recent work on fairness in machine learning has focused on various statistical discrimination criteria and how they trade off. Most of these criteria are observational: They depend only on the joint distribution of predictor, protected attribute, features, and outcome. While convenient to work with, observational criteria have severe inherent limitations that prevent them from resolving matters of fairness conclusively. Going beyond observational criteria, we frame the problem of discrimination based on protected attributes in the language of causal reasoning. This viewpoint shifts attention from "What is the right fairness criterion?" to "What do we want to assume about our model of the causal data generating process?" Through the lens of causality, we make several contributions. First, we crisply articulate why and when observational criteria fail, thus formalizing what was before a matter of opinion. Second, our approach exposes previously ignored subtleties and why they are fundamental to the problem. Finally, we put forward natural causal non-discrimination criteria and develop algorithms that satisfy them.
The paper proposes a causal view on the question of fair machine learning. Its main contribution is the introduction of causal language to the problem, and specifically the notion of "resolving variable": a variable that mediates the causal effect of a protected attribute in a way the user deems "fair". An example would be the variable "choice of departments" in the famous Simpson's paradox college-admission scenario. 
Overall I think this paper has a lot of potential, but still needs some work on clarifying many of the concepts introduced. Fairness is a difficult subject as it clearly goes beyond the math and into societal and normative discussions. As such, I am looking forward to the authors replies to my comments and questions below, hoping to have a fruitful discussion.
Pros:
1. This paper present an important contribution to the discussion on fairness, by introducing the notion of resolving variable. I have already found this term useful in discussions about the subject.
2. The paper shows how in relatively simple scenarios the proposed definitions and methods reduce to very reasonable solutions, for example the result in Theorem 2 and its corollaries. 
Cons:
1. No experimental results, not even synthetic. Other papers in the field have experiments, e.g. Hardt Price & Srebro (2016) and Chulechova (2016).
2. Lack of clarity in distinction between proxies and non-resolving variables.
3. I am not convinced of the authors view on the role of proxy variables.
4. I am not convinced that some of the proposed definitions capture the notion of fairness the authors claim to capture.
Specific comments:
1. What exactly is the difference between a proxy variable and a non-resolving variable? Does the difference lie only in the way they are used?
2. Why is the definition of proxy discrimination (def. 3) the one we want? 
Wouldn't the individual proxy discrimination be the one which more closely correlates with our ideas of fairness?
E.g., say name (variable P) affects past employment (variable X), which in turn affects the predictor. Couldn't averaging this effect over X's (even considering the do-operator version) lead to a predictor which is discriminating with respect to a specific X?
3. Why is the notion of proxy variables the correct one to control for? I think on the contrary, in many cases we *do* have access to the protected attribute (self-declared race, gender, religion etc.). It's true that conceiving intervention on these variables is difficult, but conceiving the paths through which these variables affect outcomes is more plausible. The reason gender affects employment goes beyond names: it relates to opportunities presented, different societal attitudes, and so far. The name is but one path (an unresolved path) through which gender can affect employment. 
4. The wording in the abstract: "what do we want to assume about the causal DGP", seems odd. The causal DGP exists in the world, and we cannot in general change it. Do the authors mean the causal DGP *that gives rise to a predictor R* ? 
5. Why have the mentioned approaches to individual fairness not gained traction?
6. The paper would be better if it mentioned similar work which recently appeared on arxiv by Nabi & Shpitser, "Fair Inference on Outcomes". In general I missed a discussion of mediation, which is a well known term in causal inference and seems to be what this paper is aiming at understanding. 
7. I am confused by the relation between the predictions R and the true outcomes Y. Shouldn't the parameters leading to R be learned from data which includes Y ?
8. In figure 4, why does the caption say that "we generically do not want A to influence R"? Isn't the entire discussion premised on the notion that sometimes it's ok if A influences R, through a resolving variable? 
Minor comments:
1. line 215 I think there's a typo, N_A should be N_P?