Simple alternative to reversible jump for nested models? Suppose that we have a model such that $p(y\mid k, \theta_1,\dots,\theta_{k_\text{max}})$ depends only on $k,\theta_1,\dots,\theta_k$. Hence, as $k$ assumes the values $1,\dots,k_\text{max}$, we have a family of nested models. With suitable priors for $k$ and the $\theta_j$'s, instead of using Green's reversible jump scheme, I was thinking if an iteration of the following much simpler procedure could work:

*

*Sample $k$ from the (available) full conditional distribution $k\mid y,\theta_1,\dots,\theta_{k_\text{max}}$.

*Do a Metropolis-Hastings step, sampling en bloc only $\theta_1,\dots,\theta_k$.

*Repeat.

Is this already used in the literature? If not, what is the catch? Does it sample the space of nested models incorrectly or inefficiently?

This paper Bayesian analysis of mixture models with an unknown number of components—an alternative to reversible jump methods by Matthew Stephens is somewhat along those lines

However, our approach appears the more natural and elegant in this
context, exploiting the natural nested structure of the models and
exchangeability of the mixture components. As a result we remove the
need for calculation of a complicated Jacobian, reducing the potential
for making algebraic errors

It's a bit more complicated than your setting -- the full conditional for $k$ isn't available so $k$ is modified by a birth-death process
