The Mythos of Model Interpretability
When educated people like doctors, lawyers, managers, etc., ask for &quot;interpretability,&quot; what they mean is, &quot;what are the &#x27;key variables&#x27; we can control that will help produce the outcomes we want?&quot;<p>Simpler statistical models, and particularly linear models, provide easy-to-grasp &quot;answers&quot; to that question -- for example, &quot;to get more of y, we need more of x1 and x2, and less of x3.&quot;<p>An entire generation of leaders and professionals has been trained to think this way by statistics professors and teachers who have been rigorously trained to think that way too. (For background, I highly recommend Leo Breiman&#x27;s now-classic paper, &quot;Statistical Modeling: The Two Cultures:&quot; <a href="https:&#x2F;&#x2F;projecteuclid.org&#x2F;download&#x2F;pdf_1&#x2F;euclid.ss&#x2F;1009213726" rel="nofollow">https:&#x2F;&#x2F;projecteuclid.org&#x2F;download&#x2F;pdf_1&#x2F;euclid.ss&#x2F;100921372...</a> )<p>Alas, there&#x27;s no going back.  Simpler models <i>do not work</i> for complicated tasks like, say, predicting the next best move in a game of Go, for which there are no &#x27;key variables&#x27; in the traditional sense.