How can we assume the models are exhaustive in Bayesian Model Averaging? Bayesian model averaging is justified using the law of total probability which requires the the set of models that we average over to be exhaustive. Shouldn’t  we prove that the set of models are exhaustive first?
Even if you were to claim that this marginal density is an approximation; can w e prove that there exists a set of models which are exhaustive and the set of models that we are averaging in our approximation is a random subset of that?

This can be formally viewed as conditioning on another event $C$ that asserts (i.e., another modeling assumption) that the true model is one of the models under consideration.

For simplicity, let's assume that we are interested in finding $P(A)$ for some event $A$. A true exhaustive partition is given by $B_1, B_2, \ldots B_N, B_{N+1}$. The relevant analogy here, is that we are unaware of the event $B_{N+1}$ and we (incorrectly) assume that $B_1, \ldots B_N$ is exhaustive.
That is, we are implicitly conditioning on the event $C = \bigcup_{i=1}^N B_i$, which has probability $P(C) = 1 - P(B_{N+1})$. Conditional on $C$, the probability of each event in our "partition" is now
$$P(B_i|C) = \frac{P(B_i)}{1-P(B_{N+1})}.$$
So when we try to compute the probability of $A$, we are actually computing the probability of $A$ conditional on our modeling assumption $C$.
\begin{align*}
P(A|C) &= \sum_{i=1}^N P(A|B_i, C)P(B_i|C) \\[1.2ex]
&= \sum_{i=1}^N P(A|B_i, C)\frac{P(B_i)}{1-P(B_{N+1})} \\[1.2ex]
&= \frac{1}{1-P(B_{N+1})}\sum_{i=1}^N P(A|B_i, C)P(B_i)  \\[1.2ex]
&= \frac{1}{1-P(B_{N+1})}\sum_{i=1}^N P(A|B_i)P(B_i) && \text{since $B_i \cap C = B_i$} \\[1.2ex]
\end{align*}
From here, we can easily see that
$$P(A) = P(A|C)(1-P(B_{N+1})) + P(A|B_{N+1})P(B_{N+1})$$
Thus, as long as $P(B_{N+1})$ is sufficiently small, the modeling assumption will lead us to a reasonably good approximation for the target probability.

To relate this back to the original question (and also to the other answers), there are two questions we must consider.

*

*Are the models under consideration "nearly" exhaustive? (i.e. is $P(C) \approx 1$)

*Do the models not under consideration give very different answers from the models under consideration? (i.e. is $P(A|B_{N+1}) \approx P(A|C)$)

The answers to these questions will determine the success of BMA.
