Can any probability be expressed as an expectation? In section 6.1 of the notes Stat 3701 Lecture Notes: Bayesian Inference via Markov Chain Monte Carlo (MCMC) by Charles J. Geyer, the author states

Suppose we have a probability or expectation we want to estimate. Probability is a special case of expectation: if $g$ is a zero-or-one valued function, then
$$
E\{ g(X) \} = \Pr\{ g(X) = 1 \}
$$
and any probability can be written this way. So we just consider expectations.

I would assume that $g$ in this context is the function
$$
g(X) =
\begin{cases}
1 & \ \text{if} \ \ X \in A \\
0 & \ \text{if} \ \ X \notin A
\end{cases}
$$
such that
$$
\Pr\{ g(X) = 1 \} = \Pr\{ X \in A \}
$$
where $A$ is some subset of the range (image) of $X$. Is this true for any random variable $X$?

The expectation is a Lebesgue integral with respect to some probability measure $\mu$ in $(\Omega,\mathcal{F},\mu)$,
$$\int f d\mu$$
where $f$ is a $\mathcal{F}$-measurable function, which in probability are our random variables $X$.
By construction, the Lebesgue integral of an indicator function is the measure of the set associated with that indicator,
$$ \int I_A d\mu = \mu(A), $$
thus under $(\Omega,\mathcal{F},\mathbb{P})$,
$$\mathbb{E}[I_A]=P(X\in A) = \mathbb{P}(X^{-1}(A))$$.
