When mathematical statistics outsmarts probability theory This is not a question, but it is too good to pass. I read it is originally due to Enis, Peter. "On the relation $E (X) = E [E (X∣ Y)]$." Biometrika 60, no. 2 (1973): 432-433.
Assume $Y$ has a Chi-square distribution, with one degree of freedom, so its density is
$$f_Y(y) = \frac{1}{\sqrt{2\pi}}y^{-1/2}\exp\left\{-\frac 12 y\right\},\;\;\; y \in [0,\infty). \tag{1}$$
Suppose now that we define the conditional density of another variable $X$ as
$$f_{X|Y}(x \mid y) = \frac{1}{\sqrt{2\pi}} y^{1/2}\exp\left\{-\frac 12yx^2\right\}\;\;\; x \in (-\infty, \infty). \tag{2}$$
Namely, conditional on $Y=y$, $X$ has a zero-mean Normal distribution with variance equal to $1/y$.
This is perfectly legal and valid. Note that we have
$$E(X\mid Y) = 0.$$
Then "automatically", we would conclude
$$...\implies E\big[E(X\mid Y)\big] = E(X) = 0...$$
BUT THIS IS NOT THE CASE.
By Bayes theorem for densities, the joint density is
$$f_{X,Y}(x,y) = f_{X|Y}(x \mid y)\cdot f_Y(y) = \frac{1}{2\pi}\exp\left\{-\frac 12y(1+x^2)\right\}\;\;\; (x,y) \in (-\infty, \infty) \times (0, \infty). \tag{3}$$
From this we can obtain the marginal density of $X$ by
$$f_X(x) = \int_0^{\infty}f_{X,Y}(x,y) dy = \int_0^{\infty}\frac{1}{2\pi}\exp\left\{-\frac 12y(1+x^2)\right\}\,dy$$
$$=\frac {1}{\pi}\frac{1}{1+x^2}\int_0^{\infty}\frac{1+x^2}{2}\exp\left\{-\frac 12y(1+x^2)\right\}\,dy.$$
The integral is equal to $1$ since it is an Exponential density with rate parameter $(1+x^2)/2$, so we obtain
$$f_X(x) = \frac {1}{\pi}\frac{1}{1+x^2}, \tag{4}$$
which is the standard Cauchy density, for which $E(X)$ is undefined.
So we have obtained that $E(X\mid Y)$ can exist and be finite, even when $E[X]$ does not exist.
Why has mathematical statistics outsmarted probability theory in this instance?
Because in the latter, in order to define the conditional expectation, we start by assuming a random variable $X$ for which $E(X)$ exists, and given this, $E(X\mid Y)$ is then defined -the "defining property" of the conditional expectation in this approach is exactly that its mean equals the unconditional mean. So if we were told, "let $X$ be a standard Cauchy density", we would conclude that "it follows that we cannot define a conditional expectation of it"... But we just saw that this premise of the existence of $E(X)$ is not necessary for $E(X\mid Y)$ to exist...
...the dark side of this achievement, is that it creates the following additional obligation: whenever we encounter first a conditional expectation, we cannot automatically "average over it" to obtain the unconditional expected value -the latter must be proven to exist by other means.
In other words, the existence of the unconditional expected value is only sufficient for the conditional expectation to exist, and the existence of the conditional expectation is not sufficient for the unconditional expected value to exist.
Ah, and here is a question: Any other such lovely examples?

I do not find this any more surprising than saying that if $Y \sim \mathcal N(0,1)$ then $\mathbb E\left[\frac1Y\right]$ is undefined even though $\frac1Y$ has a distribution symmetric about $0$.
So let's use this to construct an example using $Y\sim \mathcal N(0,1)$:

*

*let $X=\pm\frac1Y$ with equal probability (or $0$ in the zero-probability case of $Y=0$)

*clearly $\mathbb E\left[X \mid Y\right]=0$  so $\mathbb E\big[\mathbb E\left[X \mid Y\right]\big]=0$

*but $X$ and $\frac1Y$ have the same heavy-tailed symmetric distribution so $\mathbb E\left[X\right]$ is undefined

