Explain in layperson's terms why predictive models aren't causally interpretable Imagine that you are asked to infer some causal effect -- a change in an outcome $y$ in response to some variable $x$.  But, the person asking for this directs you to use a predictive model to do so.  Here's the setup:

*

*$x$ is confounded inasmuch as there is some unobserved $u$ that is causally linked to both $y$ and $x$.  We have a classical omitted variables bias.

*We have high dimensional covariates $\mathbf{Z}$ that are not independent of $y$ or $x$ and/or $u$

*You are asked to train a suite of predictive models -- neural networks, boosted trees, whatever -- denoted $g_i([x, \mathbf{Z}]) + \epsilon$ where $i$ indexes different models, and then select among them model $i$ that minimizes some metric of predictive skill.  RMSE, for instance.

*Based on the chosen model, you are asked to report
$$
\frac{\partial \hat{y}}{\partial x} = \frac{\partial \hat{g}_i([x, \mathbf{Z}])}{\partial x}
$$

*You know that
$$
E\left[\frac{\partial \hat{y}}{\partial x}\right] \neq \frac{\partial y}{\partial x}
$$
in the population, because the error term includes the omitted variable, so therefore
$$
\frac{\partial \epsilon}{\partial x} \neq 0 \text{  in the population, despite the fact that } \frac{\partial \hat\epsilon}{\partial x} = 0
$$
in any reasonable model $g$.

On top of omitted variables bias, there may be bias from regularization too!

*

*Further assume that you have some causal model -- say an instrumental variables regression, utilizing some suitable instrument $w$ for $x$.  It's one of the models in your suite of models, but its predictive skill in terms of cross-validated RMSE is worse than the  others.

The best model is the one that produces the consistent causal estimate, right?  But:
How would you explain this to someone in layperson's terms?
The person asking for analysis doesn't understand causal inference, and needs to be educated.  However, they don't understand math and have little attention span.  How can you effectively convey the basic point that causal methods are required, and predictive methods are inappropriate?  No math, lots of stories, pithy sentences.

Correlation does not equal causation. Predictive models using advanced techniques such as machine learning can be quite good at finding associations between predictive variables and an outcome, but this isn't the same as determining the causal relationships between those variables.
For example, as a researcher you may find a strong correlation between homelessness ($Y$) and illegal drug use ($X$) in a city, and could even state with a high degree of accuracy you can predict a person is homeless if you know they're a drug user. Can you confidently report to the city government that illegal drug use causes homelessness: $X \rightarrow Y$, and therefore reducing drug use will reduce homelessness?
No, not without either deducing or collecting more information about the causal relationship between $X$ and $Y$. Perhaps it's the reverse, homelessness causes a higher risk for illegal drug use: $X \leftarrow Y$? Or perhaps $X$ and $Y$ are not as closely related, or even completely independent, and there's in fact a third variable such as mental illness ($Z$) that causes both homelessness and illegal drug use: $X \leftarrow Z \rightarrow Y$? In both of these cases the structure of your causal inference model will have to be altered from what you might see in a typical predictive model. There are many other possibilities as well (such as mediator and collider variables) which must be accounted for or ruled out in order to draw a complete picture of the cause and effect relationships.
