Why are we using black box models in AI when we don’t need to? (2019)
I think it's important to note that <i>human</i> pattern recognition is basically black-box as well.<p>We can't "explain" how we recognize a particular person's face, for example.<p>Robust pattern recognition based on 100+'s of factors is just inherently black box.<p>Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization afterwards, but psychologists have shown that this is often a bunch of post-hoc fiction.<p>Black box doesn't necessarily mean bad. I think the relevant question is: how do we ensure machine learning is trained and verified in ways that don't encode bias, and only used with "intended" inputs, so a model isn't being trusted to make predictions on things it wasn't trained for?<p>And also: when do we want to decide to use alternatives like decision trees instead -- because despite being less accurate, they can have designers who can be held legally accountable and liable, which can be more important in certain situations?