Simple alternative to reversible jump for nested models? Suppose that we have a model such that $p(y\mid k, \theta_1,\dots,\theta_{k_\text{max}})$ depends only on $k,\theta_1,\dots,\theta_k$. Hence, as $k$ assumes the values $1,\dots,k_\text{max}$, we have a family of nested models. With suitable priors for $k$ and the $\theta_j$'s, instead of using Green's reversible jump scheme, I was thinking if an iteration of the following much simpler procedure could work:

*

*Sample $k$ from the (available) full conditional distribution $k\mid y,\theta_1,\dots,\theta_{k_\text{max}}$.

*Do a Metropolis-Hastings step, sampling en bloc only $\theta_1,\dots,\theta_k$.

*Repeat.

Is this already used in the literature? If not, what is the catch? Does it sample the space of nested models incorrectly or inefficiently?

The alternative to reversible jump is to go for the respective posterior probabilities of the different models,
$$\mathbb P(\mathfrak M_k|\mathbf x)\propto
\pi(\mathfrak M_k) \underbrace{\int_{\Theta_k} \pi_k(\theta_k)f_k(\mathbf x|\theta_k)\,\text d\theta_k}_\text{$\mathfrak e_k$: evidence on $\mathfrak M_k$}$$
which can be approximated from simulations of $\pi_k(\theta_k|\mathbf x)$. Hence a marginal MCMC scheme could proceed by exploring the set of models (or model indices) as follows:
At iteration $t$, with current model index $k^{(t)}$

*

*make a proposal $k^\prime\sim q(\cdot|k^{(t)})$

*compute an unbiased estimator $\hat{\mathfrak e}_{k^\prime}$, e.g. by running some MCMC simulations targetting $\pi_{k^\prime}(\cdot|\mathbf x)$

*accept the move with probability $$\dfrac{\pi(\mathfrak M_{k^\prime})\hat{\mathfrak e}_{k^\prime}}{\pi(\mathfrak M_{k^{(t)}})\hat{\mathfrak e}_{k^{(t)}}}\,\dfrac{q(k^{(t)}|k^\prime)}{q(k^\prime|k^{(t)})}$$
In general, it is not possible to use solely simulations from the encompassing (or largest) model, since the posterior measures on the $\theta_k$'s are supported by sets of measure zero wrt the encompassing measure. This is why, even if model $\mathfrak M_k$ corresponds to a subset of $(\theta_1,\dots,\theta_{k_\text{max}}$ it must be treated as a completely different object, with a separate prior density.
