Explain in layperson's terms why predictive models aren't causally interpretable Imagine that you are asked to infer some causal effect -- a change in an outcome $y$ in response to some variable $x$.  But, the person asking for this directs you to use a predictive model to do so.  Here's the setup:

*

*$x$ is confounded inasmuch as there is some unobserved $u$ that is causally linked to both $y$ and $x$.  We have a classical omitted variables bias.

*We have high dimensional covariates $\mathbf{Z}$ that are not independent of $y$ or $x$ and/or $u$

*You are asked to train a suite of predictive models -- neural networks, boosted trees, whatever -- denoted $g_i([x, \mathbf{Z}]) + \epsilon$ where $i$ indexes different models, and then select among them model $i$ that minimizes some metric of predictive skill.  RMSE, for instance.

*Based on the chosen model, you are asked to report
$$
\frac{\partial \hat{y}}{\partial x} = \frac{\partial \hat{g}_i([x, \mathbf{Z}])}{\partial x}
$$

*You know that
$$
E\left[\frac{\partial \hat{y}}{\partial x}\right] \neq \frac{\partial y}{\partial x}
$$
in the population, because the error term includes the omitted variable, so therefore
$$
\frac{\partial \epsilon}{\partial x} \neq 0 \text{  in the population, despite the fact that } \frac{\partial \hat\epsilon}{\partial x} = 0
$$
in any reasonable model $g$.

On top of omitted variables bias, there may be bias from regularization too!

*

*Further assume that you have some causal model -- say an instrumental variables regression, utilizing some suitable instrument $w$ for $x$.  It's one of the models in your suite of models, but its predictive skill in terms of cross-validated RMSE is worse than the  others.

The best model is the one that produces the consistent causal estimate, right?  But:
How would you explain this to someone in layperson's terms?
The person asking for analysis doesn't understand causal inference, and needs to be educated.  However, they don't understand math and have little attention span.  How can you effectively convey the basic point that causal methods are required, and predictive methods are inappropriate?  No math, lots of stories, pithy sentences.

I don't think you even need to posit a covariate adjustment set $\textbf{Z}$ nor the indexation of black-box models to convey in layman terms the main point. Assume the following:

*

*$y$ is number of people drowning in a given month in a given city

*$x$ is number of ice-cream sold in a given month in a given city

*$u$ is temperature in a given month in a given city, the unobserved confounder

$x$ will be highly predictive of $y$, and most likely a model just using $x$ as a predictor will outperform models that use noisier measurements of the real causes or their instrumental variables.
Clearly, the best predictive model is not necessarily the one that gives the most consistent causal estimate.
