The problem is not so much about whether predictions are biased. It&#x27;s a problem of whether the algorithm is <i>reinforcing pre-existing biases</i>. In other words, it&#x27;s about taking short-term losses in return for long-term gains.<p>Suppose a certain group of people have experienced some kind of historical discrimination, and as a result people in that group are starting from a disadvantaged position (wealth, etc) compared to the rest of the population. It may well be an unbiased prediction that such people will be a greater credit risk, etc. However, by making decisions solely on the basis of this, although the bank saves some money in the short-term, it reinforces the cycle of discrimination.<p>If your goal is to <i>improve society in the long-term</i>, making decisions based on these <i>short-term predictors</i> is not a desirable thing to do.