How to prove that, in finite sample, Bayesian posterior is more informative than the prior? Suppose there is a space of possible models $\theta \in \Theta$, and that we can generate i.i.d. data $\{x_1, x_2,...\}$ from the true model.
Asymptotically, the Schwartz Theorem shows that Bayesian inference is consistent: the posterior will cluster around the true model.
However, can the same be shown in finite sample? That is, can we show that, in some appropriate sense, the posterior after one data observation $P(\theta | x)$ is (on average) more informative than the prior $P(\theta)$?
When the number of models is finite, a more concrete version of this question is: can we show that $E(P(\theta^\text{true} | x)) > P(\theta^\text{true})$?

Okay, I think I have an answer now. Consider the case where both $\theta$ and $x$ can only take finite values. We want to show that $E(P(\theta^\text{true} | x)) > P(\theta^\text{true})$. Notice that:
\begin{align*}
E(P(\theta^\text{true} | x)) & = E\left( \frac{P(x | \theta^\text{true}) P(\theta^\text{true})}{P(x)} \right) \\
& = P(\theta^\text{true}) \cdot \underbrace{E\left( \frac{P(x | \theta^\text{true})}{P(x)} \right)}_\text{Y}
\end{align*}
Thus, we just need to show that $Y > 1$. Let's define $\epsilon(x)$ such that $P(x | \theta_T) = P(x) + \epsilon(x)$. Clearly, due to the property of probability mass functions, $\sum_x \epsilon(x) = 0$. Using this notation, it is easy to see that:
\begin{align*}
Y & = \sum_x \frac{P^2(x | \theta^\text{true})}{P(x)} \\
& = \sum_x \frac{[P(x) + \epsilon(x)]^2}{P(x)} \\
& = \sum_x \left\{ P(x) + 2 \epsilon(x) + \frac{\epsilon^2(x)}{P(x)} \right\} \\
& = 1 + 0 + \sum_x \frac{\epsilon^2(x)}{P(x)} > 1
\end{align*}
where the last step assumes that $\epsilon(x') \ne 0$ for some $x'$. That is, the different models in $\Theta$ cannot have identical likelihoods.
