Explain in layperson's terms why predictive models aren't causally interpretable Imagine that you are asked to infer some causal effect -- a change in an outcome $y$ in response to some variable $x$.  But, the person asking for this directs you to use a predictive model to do so.  Here's the setup:

*

*$x$ is confounded inasmuch as there is some unobserved $u$ that is causally linked to both $y$ and $x$.  We have a classical omitted variables bias.

*We have high dimensional covariates $\mathbf{Z}$ that are not independent of $y$ or $x$ and/or $u$

*You are asked to train a suite of predictive models -- neural networks, boosted trees, whatever -- denoted $g_i([x, \mathbf{Z}]) + \epsilon$ where $i$ indexes different models, and then select among them model $i$ that minimizes some metric of predictive skill.  RMSE, for instance.

*Based on the chosen model, you are asked to report
$$
\frac{\partial \hat{y}}{\partial x} = \frac{\partial \hat{g}_i([x, \mathbf{Z}])}{\partial x}
$$

*You know that
$$
E\left[\frac{\partial \hat{y}}{\partial x}\right] \neq \frac{\partial y}{\partial x}
$$
in the population, because the error term includes the omitted variable, so therefore
$$
\frac{\partial \epsilon}{\partial x} \neq 0 \text{  in the population, despite the fact that } \frac{\partial \hat\epsilon}{\partial x} = 0
$$
in any reasonable model $g$.

On top of omitted variables bias, there may be bias from regularization too!

*

*Further assume that you have some causal model -- say an instrumental variables regression, utilizing some suitable instrument $w$ for $x$.  It's one of the models in your suite of models, but its predictive skill in terms of cross-validated RMSE is worse than the  others.

The best model is the one that produces the consistent causal estimate, right?  But:
How would you explain this to someone in layperson's terms?
The person asking for analysis doesn't understand causal inference, and needs to be educated.  However, they don't understand math and have little attention span.  How can you effectively convey the basic point that causal methods are required, and predictive methods are inappropriate?  No math, lots of stories, pithy sentences.

I think this explanation is best approached sequentially. Start with a simple story:

When my dog Winston wags his tail, that indicates he is happy. For instance, he never wags it at the vet, wags it a bit when I get his leash, and wags a whole lot when I also grab a tennis ball. But if I wag Winston's tail for him, it usually has the opposite effect.

In other words, a "wagging tail" is a good predictor of my dog's state of mind, but I cannot use this knowledge to make him happy (except as a kind of proxy variable in experiments). Here the causality is pretty straightforward, so the contrast between prediction and cause is stark.
The next parable is both more realistic and closer to home:

If you look at the performance of salespeople at my company, the ones
with expensive cars are the most productive. While it is possible that
clients find luxury cars impressive, and that makes selling to them easier, our sales happen over the phone, so it is unlikely that giving our salespeople nice cars will boost revenue (unless there is a promise to let the customer take the Porsche for a spin after the deal is sealed).

The causality goes the other way here, though there is a slight potential for the correlation between sales and cars to be causal.
Now for another example:

It is evident that people who have our app installed on their phone buy more than folks that only shop in person and/or through the website. The app sends notifications and it makes it much easier to buy stuff with just a click. But people don't just install the app for no reason. They do it because they expect to buy more, which the app makes more convenient, so comparing customers with and without the app is like comparing apples and orangutans. They are very different people.

Here there is causality in both directions, but arguably the high intent $ \rightarrow$ expenditures mechanism dominates the app install $\rightarrow$ expenditures. When a causal explanation works in both directions, you can usually settle the debate with an experiment to see which one is the most important.
The real world is vastly more complicated than these fairly simple stories, and our intuition can often lead us astray at a great cost. Here are two more good examples from industry of mistaking correlation for causation:

*

*Ascarza, Eva. Retention Futility: Targeting High-Risk Customers Might Be Ineffective. Journal of Marketing Research (JMR) 55, no. 1 (February 2018): 80–98.

*Blake, T., Nosko, C. and Tadelis, S. (2015), Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment. Econometrica, 83: 155-174. https://doi.org/10.3982/ECTA12423
