That machine learning algorithms can demonstrate bias is well-documented by now. This work confronts the challenge of bias mitigation in fully-connected feedforward neural nets through the lens of multiobjective optimisation. Specifically, recognising that fairness and accuracy are competing objectives, the proposed methodology uses techniques from multiobjective optimisation to ascertain the fairness-accuracy landscape of a neural net classifier. Along the way, a new causal notion of fairness is introduced that is particularly suited to giving a nuanced treatment when data is collected under unfair practices. In particular, special attention is paid to subjects whose covariates could appear with substantial probability in either value of the sensitive attribute. Experimental results suggest that the proposed method produces neural net classifiers that distribute evenly across the Pareto front of the fairness-accuracy space and is more efficient at finding non-dominated points than an adversarial approach.
This paper proposes a method to approximate the "fairness-accuracy landscape" of classifiers based on neural networks.
The key idea is to set up a multi-dimensional objective, where one dimension is about prediction accuracy and another 
about fairness. The fairness component relies on a  definition of fairness based on causal inference, relying on the 
idea that a sensitive attribute should not causally affect model predictions.
I found the causal idea intriguing, since it makes sense that we don't want a sensitive attribute to have a causal effect.
However, there may be several problems with this approach:
1) For a causal estimate to be valid we need several assumptions. For example, we need A (the sensitive attribute) 
to be independent of potential outcomes conditional on X --- the so-called "unconfoundedness assumption" in causal inference.
We also need "positivity", i.e., that 0< P(A=1|X) <1. 
These assumptions are not discussed in the paper. Furthermore, the particular context of the paper, where the treatment is actually an immutable characteristic, makes such discussion much more subtle. 
What will we do, for instance, if there are no A=1 in the sample when X = ...?
2) The authors seem to assume that the propensity score model is well specified. This can be tested, e.g., using [1].
What do we do when this fails? 
3) Why do we want U to be small, i.e., why do we want the causal effect of A to be small, is never justified.
In particular, its relation to "fairness" is never fleshed out, but just assumed to be so.
This can be problematic when, say, we are missing certain important X that are important for A. 
Then, there will be a measurable causal effect of A on h(). 
Some other problems:
- What is the reason for focusing on 'neural classifiers'? There is nothing specific in the method or analysis 
that relates to neural networks, except for the use of the causal estimand in a 'hidden layer'.
- In the Introduction, the authors could cite the works of Amartya Sea, etc., on fairness. 
Certainly the study of fairness problems did not start in 2016.
- What exactly is a "sensitive attribute"? If we don't want to bias our predictions, then why include it in the analysis?
- It is unclear what is new and what is related work in page 3.
- Sec. 4: The claim that "causal inference is about situations where manipulation is impossible" discards 
voluminous work in causal inference through randomized experiments. In fact, many scientists would 
agree that causal inference is impossible without manipulation.
- As mentioned above, why this particular estimand leads to more "fairness" is never explained.
- Do we need a square or abs value in Eq (5)?
- The experimental section is weak and does not illustrate the claims well. 
   It would be important to explain the choice of the particular causal estimand, the choice of the hidden layer to put the estimand in, to explore the choice of the objective, and so on. Currently, none of these choices/design aspects are being investigated.
[1] "A specification test for the propensity score using its distribution conditional
on participation" (Shaikh et al, 2009)