What if there is no true data-generating process? I've been engaging in a number of forecasting efforts recently, and have rediscovered a well-known truth: That combinations of different forecasts are generally better than the forecasts themselves. In particular, the unweighted mean of forecasts is typically better than any of the averaged forecasts. So far, in my own work, I have not encountered any exceptions to this except when the data is artificially generated from simple models.
I was, and remain, flabbergasted by this. Why should averaging models based on entirely inconsistent assumptions generate anything but nonsense? Why is the unweighted average of the best model with relatively inferior models usually better than the best? Why do we seem to get most of the benefits of sophisticated ensemble methods from the unweighted mean?
I always thought that the modeling process was intended to find the model that most nearly approximated the underlying reality, imperfectly, of course, but still assuming that there would always be the best model given specified constraints of parsimony, data availability, and the like. To me, the fact that the unweighted mean of a more-or-less arbitrary collection of model types (that experience has taught us are pretty good) does not suggest that the true model is roughly the mean of the constituent models---that would be absurd.
Instead, it suggests that there is no true data-generating process that can be approximated by any standard estimating technique, however sophisticated. The data may be generated as some complex summation or composite of many, many agents or sub-processes, each of which or who embodies a unique complex of causal forces, perhaps including multiple layers of non-linear feedback. Perhaps they are influenced or entrained by common exposure to forces that you as a modeler will never see, like the boss's mood or the ionization level in the air or irrational remnants of historical institutional structures that persist and still affect decisions.
You see this in other ways too. For example, sometimes the theory is utterly unambiguous about which models are to be preferred. It is, for example, entirely clear that most macroeconomic variables modeled by VARs or VECMs should be logged or log-differenced for multiple compelling reasons, both statistical (i.e. to avoid heteroskedasticity, to linearize any trend present), and economic. Except when you actually run such models, the opposite is true. I have no idea why.
My question is this. Has anyone found a way of formalizing the belief that processes we strive to understand have no data-generating process that we can capture in a standard mathematical model? Has anyone attempted to describe the foundations of statistics based on such a formalization -- a statistic in which all models are unavoidably misspecified? If so, does it have any known implications for hypothesis testing, and the sort of test-and-redesign process that constitutes normal workflow for a statistician or a data scientist? Should we be multiplying models earlier in the analysis process? If so, how? Should we be choosing which models to aggregate based on some principle other than the quality of fit with a complexity penalty, or some model comparison test like AIC? As things are designed ultimately to be input to ensembles, should we prioritize models that give different predictions, rather than good predictions? Is there a principled way to make such trade-offs?
And if this is the norm, why isn't it in any of the six widely-used introductory statistics texts I went through when composing this post?

Have you heard the "all models are wrong, but some are useful" quote? It's one o the most famous quotes in statistics.
Let's use human language as an example. What you say, is a result of many parallel and concurring processes. It is influenced by the rules governing the language, your fluency in the language, educational background, the books you've read in your lifetime, cultural factors, context, whom you're talking to, psychological and physiological factors influencing you at the moment of speaking, and many, many more things, and you may be quotig or misquoting someone who was influenced by them in the past, etc. There's no a function, process, or distribution that "generated" the words that came out of your mouth.
Playing an Advocatus Diaboli, now think of forecasting weather. It is hard, because weather is influenced that many interacting factors. Weather is a chaotic system. But maybe the system as a whole can be thought as a process the generates the weather?
It's a philosophical discussion. It's also an unnecessary one, at least form a practical point of view. We don't really need to believe that there's a distribution or process that generates our data. It's a mathematical abstraction. We wouldn’t be able to talk about statistical properties of estimators such as bias and variance (to give only one example), without introducing some abstract, mathematical objects for the things that are modeled. We are using mathematical functions to approximate something, this something needs also to be considered as a function, so can it be discussed in mathematical terms. We are not claiming that there exists a process that "generates" the data for us, we are just using an abstract concept to talk about it.
So yes, ale models are misspecified, wrong. They are only approximations. The "things" they approximate are just abstract concepts. If you want to really go all the way to the rabbit hole, there is no such things as sound, colors, wind, or trees, or us. We are just particles surrounded by other particles and we assign some meanings to groups particles that at a particular moment stay close to each other, but do those things exist? Maybe should we be building particle-level models of reality? A related xkcd below.

