Consider your drug conviction example, and lets say the algorithm is linear regression so I can give a very simple explanation.<p>The model will then be FICO = stuff + (-1) x (drug conviction).<p>I.e., your FICO score is lowered by 1 point if you&#x27;ve been convicted of drug use.<p>If you are correct, some enterprising quant can take your hypothesis. He can then rerun his regression with (drug conviction, black) and (drug conviction, white) as variables. If your hypothesis is correct the result will be:<p>FICO = stuff + (-0.5) x (drug conviction, black) + (-1.5) x (drug conviction, white).<p>This is because the bias in drug convictions makes it less predictive for black people.<p>If you are wrong the coefficients will be equal [1]. If you are right he just made millions of dollars for the bank he works for and probably added $100k to his bonus.<p>What you are discussing is strictly a statistics problem.<p>[1] For examples of this type of analysis on social indicators (I don&#x27;t know of any on credit allocation), see these papers: 
<a href="https:&#x2F;&#x2F;www.rand.org&#x2F;content&#x2F;dam&#x2F;rand&#x2F;www&#x2F;external&#x2F;labor&#x2F;seminars&#x2F;adp&#x2F;pdfs&#x2F;adp_ajph.pdf" rel="nofollow">https:&#x2F;&#x2F;www.rand.org&#x2F;content&#x2F;dam&#x2F;rand&#x2F;www&#x2F;external&#x2F;labor&#x2F;sem...</a> <a href="http:&#x2F;&#x2F;egov.ufsc.br&#x2F;portal&#x2F;sites&#x2F;default&#x2F;files&#x2F;anexos&#x2F;33027-41458-1-PB.pdf" rel="nofollow">http:&#x2F;&#x2F;egov.ufsc.br&#x2F;portal&#x2F;sites&#x2F;default&#x2F;files&#x2F;anexos&#x2F;33027-...</a> <a href="http:&#x2F;&#x2F;ftp.iza.org&#x2F;dp8733.pdf" rel="nofollow">http:&#x2F;&#x2F;ftp.iza.org&#x2F;dp8733.pdf</a>