Enhancing the Accuracy and Fairness of Human Decision Making
Societies often rely on human experts to take a wide variety of decisions affecting their members, from jail-or-release decisions taken by judges and stop-and-frisk decisions taken by police officers to accept-or-reject decisions taken by academics. In this context, each decision is taken by an expert who is typically chosen uniformly at random from a pool of experts. However, these decisions may be imperfect due to limited experience, implicit biases, or faulty probabilistic reasoning. Can we improve the accuracy and fairness of the overall decision making process by optimizing the assignment between experts and decisions? In this paper, we address the above problem from the perspective of sequential decision making and show that, for different fairness notions in the literature, it reduces to a sequence of (constrained) weighted bipartite matchings, which can be solved efficiently using algorithms with approximation guarantees. Moreover, these algorithms also benefit from posterior sampling to actively trade off exploitation-selecting expert assignments which lead to accurate and fair decisions-and exploration-selecting expert assignments to learn about the experts' preferences. We demonstrate the effectiveness of our algorithms on both synthetic and real-world data and show that they can significantly improve both the accuracy and fairness of the decisions taken by pools of experts.
## Updates after feedback/discussion ## The theoretical framework proposed in this paper is interesting and lends itself to numerous extensions that could be pursued in future work.  Overall I think the authors convincingly demonstrate that their approach can improve utility over the case of random assignment.  The evidence for fairness improvement might run into opposition as the assignment strategy may be viewed as discriminatory in practice.  This would be worth further discussion in the paper.  It would also be interesting to see more examples of what happens when there aren't all that many more decision makers than items.  A typical situation is one where there are more individuals than judges, so each judge needs to be used each round.  However, not all individuals are expected to be processed in each round.    I also think the authors can do more to provide motivation and intuition for their strategy.  Here's my interpretation of the overall approach.  The result for the unconstrained optimization setting can be interpreted as approximating the rule in equation (3) through a strategic assignment of judges to individuals. The rule in equation (3) applies the same threshold to everyone regardless of their group. This rule holds everyone, regardless of group, to the same optimal threshold. One can interpret the suboptimality of random assignment as arising from a (judge, individual) combination that differs from the optimal thresholding rules in equations (3). There are two reasons for why the random assignment rule may be suboptimal. (i) If judges are systematically biased against some group, the random assignment rule would effectively apply a higher standard to one group than another. We know that a single common threshold rule is optimal, so random assignment is suboptimal. (ii) Even if judges don't use group-specific thresholds, random assignment may fail to agree with thresholding at the optimal threshold c in equation (3). We can interpret the approach outlined in the paper as bringing us closer to the rule in equation (3) (equal standard) in the unconstrained case. We can interpret the results for the constrained case as bringing us closer (than random assignment) to the rule in equation (4), that optimally holds groups to different standards.  ## Original review ## Overview:  This paper considers the following problem. Suppose that there are m judges available to make decisions about n < m individuals (per round).  Individuals are described by (X, Z, Y), where X are features, Z is a group indicator (binary), and Y is an outcome.   Judges make decisions to maximize immediate utility by thresholding the common (and assumed known) conditional probability function p(x, z) = P(Y = 1 | X, Z) at judge-specific and group-specific thresholds.  What is the optimal way of assigning individuals to judges in each round so as to maximize utility subject to fairness (DI, TPR, TPR/FPR) constraints?  The authors show that the unconstrained problem can be solved efficiently as a bipartite matching between judges and individuals.  The constrained problem  can be reduced to bounded color matching. Theoretical results are provided showing that posterior sampling can achieve sublinear regret in the cases where the judge-specific thresholds are unknown and need to be estimated across rounds.  Empirical evaluations demonstrate performance of the proposed method in comparison to the more common strategy of random assignment of cases to judges.    Comments:   Overall I think this paper does a pretty good job from a theoretical standpoint of analyzing a toy problem that has some relevance to practice.   The authors readily admit that assuming all judges have access to the the same true probability function for decision making is highly idealized.  I wish the authors did more in either the discussion of the theory or the empirical results to more clearly characterize where the gains are coming from.  In terms of practical implications, the most relevant empirical comparison is between random assignment and the unknown thresholds case.  As I elaborate on below, I would like the authors to expand on this comparison to clarify their ideas.    - While DI as in (11) depends on there being 2 groups, it seems like for a more general definition of DI the results generalize to to non-binary Z.  Is this the case?  - The optimal threshold decision rules have the form given in (3) and (4) only if you assume that the probability function p(x,z) has positive density (see original Corbett-Davies paper).  If you have point masses in p(x,z), the optimal rule winds up being a randomized one.    - Discussion of COMPAS data: Individuals in bail proceedings are considered "defendants", not "offenders".  They have not yet been convicted of any crimes, and thus have not formally "offended".    - What is the precise definition of "Disparate impact" used in the experiments section?   - Your definition of “bias” in the sense of group-specific thresholds isn’t consistent with the discussion of fair classification/optimal threshold rules.  i.e., the optimal fair decision rule under those fairness constraints would have the judges applying group-specific thresholds.  So in what sense is setting unequal thresholds in your simulation “biased”?    - How does the performance degrade with the fraction of “biased” judges, and the degree of bias?  You compare in the COMPAS example for one “bias” level and 0 vs 50% “biased” judges, but I’m not sure if this is presenting a complete picture.  In what situations will the proposed strategy lead to improvement?    - When presenting the bar charts for the 50% biased case, how are infeasible solutions being treated?  Are you reporting the DI/utility omitting any infeasible solutions?  How big are the error bars on the utilities?  - By what mechanism is the DI being reduced in the experiments?  Is it because release rates are changing considerably across the Random, Optimal, Known and Unknown scenarios?  I’m especially interested in the answer to this question for the Random vs. Unknown scenario case.  If “release” rates are changing significantly, I don’t feel that the empirical evaluation is presenting an apples-to-apples comparisons.