&gt; Facebook has really, really good people working on machine learning.<p>Being good at machine learning does not imply they are also good evaluating how their ideas will impact society, nor does it imply they don&#x27;t have other motives (intentionally or not).<p>Unfortunately, too many people only consider the intended results when they create something, and rarely consider the larger impact their creation would have on other aspects of society.<p>&gt; more accurate<p>It&#x27;s questionable if this is &quot;accuracy&quot;. They are doing very <i>precise</i> calculations, but the result is still going to depend on the assumptions in the design (e.g. that you can infer anything about default risk from someone&#x27;s online &quot;friends&quot;), and the quality of the input data.<p>However, those technical concerns are not the main problem, which is <i>redlining</i>. Laws are slow in responding to changes, which allowed a lot of racists to maintain <i>de facto</i> segregation by simply claiming that someone &quot;didn&#x27;t qualify&quot; for some loan. This was easy, because the complex nature calculating someone&#x27;s default risk makes it easy to hide something that &quot;just happens&quot; to target a certain group of people. This becomes even easier (and provides more cover) when it starts to feedback on itself: (i.e. you&#x27;re a risk because you&#x27;re from $bad_area, but you have to live in $bad_area because we denied loans in the past). I highly recommend reading Ta-Nehisi Coates essay that explains this in far more detail[1].<p>Using a social-media &quot;friends list&quot; to judge loan risk is data with a very poor signal-to-noise ratio. That same &quot;friends list&quot; is also data that can be easily map to race or class, just like &quot;current address&quot; did for the lenders that redlined segregated areas to limit &quot;mud people&quot;[2] to subprime &quot;ghetto loans&quot;[2]. Which of these sounds like a more likely cause? Even if the developers at Facebook believe this is about &quot;more accurate loans&quot;, <i>someone</i> has other plans.<p>The complexity of modern technology has made it even easier to hide the real decision-making process, and machine learning techniques are the worst, where &quot;I&#x27;m not denying you a loan, the Magic Machine is! (even though I designed it and decided which data it gets to see)&quot;. These tools can certainly be useful, but it is incredibly important that you&#x27;re not just creating a proxy (both software and legal) that hides the underlying decision making process.<p>As you say, the government absolutely has a role here, but governments are slow to change. In the meantime, it is important for the people implementing to remember that <i>code is law</i>[3].<p>&gt; actuaries already use all sorts of information<p>Which is why some of them have had to pay discrimination lawsuit settlements in the $100M-300M range in recent years, with more lawsuits pending.<p>[1] <a href="http:&#x2F;&#x2F;www.theatlantic.com&#x2F;magazine&#x2F;archive&#x2F;2014&#x2F;06&#x2F;the-case-for-reparations&#x2F;361631&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.theatlantic.com&#x2F;magazine&#x2F;archive&#x2F;2014&#x2F;06&#x2F;the-case...</a><p>[2] These terms are from the affidavit[1] of Wells Fargo loan officers<p>[3] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Code_and_Other_Laws_of_Cyberspace" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Code_and_Other_Laws_of_Cybersp...</a>