Explain in layperson's terms why predictive models aren't causally interpretable Imagine that you are asked to infer some causal effect -- a change in an outcome $y$ in response to some variable $x$.  But, the person asking for this directs you to use a predictive model to do so.  Here's the setup:

*

*$x$ is confounded inasmuch as there is some unobserved $u$ that is causally linked to both $y$ and $x$.  We have a classical omitted variables bias.

*We have high dimensional covariates $\mathbf{Z}$ that are not independent of $y$ or $x$ and/or $u$

*You are asked to train a suite of predictive models -- neural networks, boosted trees, whatever -- denoted $g_i([x, \mathbf{Z}]) + \epsilon$ where $i$ indexes different models, and then select among them model $i$ that minimizes some metric of predictive skill.  RMSE, for instance.

*Based on the chosen model, you are asked to report
$$
\frac{\partial \hat{y}}{\partial x} = \frac{\partial \hat{g}_i([x, \mathbf{Z}])}{\partial x}
$$

*You know that
$$
E\left[\frac{\partial \hat{y}}{\partial x}\right] \neq \frac{\partial y}{\partial x}
$$
in the population, because the error term includes the omitted variable, so therefore
$$
\frac{\partial \epsilon}{\partial x} \neq 0 \text{  in the population, despite the fact that } \frac{\partial \hat\epsilon}{\partial x} = 0
$$
in any reasonable model $g$.

On top of omitted variables bias, there may be bias from regularization too!

*

*Further assume that you have some causal model -- say an instrumental variables regression, utilizing some suitable instrument $w$ for $x$.  It's one of the models in your suite of models, but its predictive skill in terms of cross-validated RMSE is worse than the  others.

The best model is the one that produces the consistent causal estimate, right?  But:
How would you explain this to someone in layperson's terms?
The person asking for analysis doesn't understand causal inference, and needs to be educated.  However, they don't understand math and have little attention span.  How can you effectively convey the basic point that causal methods are required, and predictive methods are inappropriate?  No math, lots of stories, pithy sentences.

The basic problem is that a non-causal predictive model may fail if used for interventions. Start with a very simple example where an important part of the real world has:
$$ Cause \rightarrow \mathit{Effect} \rightarrow Measurement $$
Because we imagine Measurement is nearly perfect, the best predictive model will prefer it to a noisier Cause, but Measurement can only be known after Effect, so it cannot inform policies or interventions. Changing Measurement by some other means won't change Effect at all.
 ~ * ~ Possibly that's sufficient, but more detail below. ~ * ~

If Measurement is something like income or health or quality, you can of course measure it ahead of time, but then it's not really the right measurement, because it hasn't taken Effect into account yet. The predictions won't be as good. Here you would have done better to use Cause, if known.
Often Cause is not known, and Measurement could be as good a predictor as you like. In the ultimate example, we have entangled particles:
$$ \mathit{Spin\ 1} \leftarrow \mathit{Hidden\ Cause} \rightarrow \mathit{Spin\ 2} $$
In this case, Spin 2 is a perfect predictor of Spin 1, but it can't ever be used to change Spin 1: they both simply reflect a hidden common cause. In a less extreme example, ice-cream sales and drownings are both caused by heat, but also other things:
$$Other1 \rightarrow Sales \leftarrow \mathbf{Heat} \rightarrow Drownings \leftarrow Other2$$
We can't reduce drownings by banning ice-cream, and we can't do much about Heat, however predictive, but we can make policies about Other2. (And ice-cream sales could be an early warning allowing us to act.)
Usually there is no one perfect Measurement, so we manufacture one: our predictive model cleverly combines Many things. But even if it correctly accounts for dependencies among Many things, it doesn't help policy or interventions if Many things are downstream of Effect.
If Measurement is downstream of both Cause and Effect, then the model can fail in stranger ways, esp. if it was cleverly trying to control things statistically.
