Bayesian updates for Dirichlet-multinomial with Gamma prior Let
$$
 \begin{aligned}
  X_i &\sim \text{Dir-multinom}(X\mid\lambda)\\
  \lambda_{j} &\sim \text{Gamma}(\lambda_j\mid\alpha,\beta)\\
\end{aligned}
$$
where $i$ iterates over observations, $j$ iterates over categories.
I'm trying to perform inference on $\lambda$.  My thought was to calculate the latent $\pi$, such that the model becomes:
$$
\begin{aligned}
X_i &\sim \text{Multinomial}(X_i\mid\pi_i)\\
\pi_i &\sim \text{Dirichlet}(\pi\mid\lambda)\\
\lambda_j &\sim \text{Gamma}(\lambda_j\mid\alpha,\beta)
\end{aligned}
$$
Then posterior updating for $\pi$ in the Gibbs sampler becomes:
$$
\pi_i \mid X_i, \lambda \sim \text{Dirichlet}(\pi_i\mid \lambda + X_i)
$$
Which is frequently sampled as
$$
\rho_{ij}\mid X_{ij}, \lambda_j \sim \text{Gamma}(\rho_{ij}\mid \lambda_{j} + X_{ij}, 1)
$$
then Gamma $\rho_i$ is reduced to Dirichlet $\pi_i$ by projecting onto the simplex (divide by the sum). With these $\rho_i$'s, I should be able to produce estimates for $\lambda$ as owing to the second model formulation, the full conditional for $\lambda$ is proportional to
$$
f(\lambda_j\mid\rho) \propto \prod_{i = 1}^{n}\text{Gamma}(\rho_{ij}\mid\lambda_j, 1)\times \text{Gamma}(\lambda_j\mid\alpha,\beta)
$$
then sampling $\lambda$ using Metropolis Hastings.
However, when I implement this sampler, my estimates for $\lambda$ and subsequently $\rho$ just grow larger the longer I run the sampler.
I suppose I should mention that I can use the first model formulation; updating is slower as it can't be done marginally.  Is there any valid reason the second model can not be implemented?  I have reason for wanting the latent $\pi$'s/$\rho$'s.

Let's look carefully at the conditional distribution of $\lambda$ : Using the original parametrization
$$ P(\lambda | \pi ) \propto \prod_i \text{Dirichlet}(\pi_i | \lambda) \times \prod_j \text{Gamma}(\lambda_j|\alpha,\beta) \\ = \left(\frac{1}{B(\lambda)}\right)^n \prod_{i,j} \pi_{ij}^{\lambda_j-1} \times \prod_j \text{Gamma}(\lambda_j|\alpha,\beta)$$
We can see immediately that the $\lambda_j$'s are not independent conditional on $\pi_{ij}$  (due to to the $B(\lambda)$ factor).
Alternatively using the $\rho$ parametrization we can express the model as :
$$ \rho_{ij} \sim \text{Gamma}(\lambda_j,1)$$
$$ X_i \sim \text{Multinomial}\left(\frac{\rho_i}{\sum_j \rho_{ij}} \right) $$
However now the posterior probability of $\rho$ is
$$P(\rho|X,\lambda) \propto \prod_{ij} \left(\frac{\rho_{ij}}{\sum_j \rho_{ij}}\right)^{X_{ij}} \times \text{Gamma}(\rho_{ij} | \lambda_j,1) \\ = 
\prod_i \frac{1}{(\sum_j \rho_{ij})^{\sum_j X_{ij}}} \prod_j \rho_{ij}^{X_{ij}} \times \text{Gamma}(\rho_{ij}|\lambda_j,1)$$
which is not a product of independent Gamma distributions (due to the $\sum_j \rho_{ij}$ term).
So although now the $\lambda_j$'s are independent conditional on $\rho_{ij}$, the $\rho_{ij}$'s themselves are not independent conditional on the data. Therefore in either way we can't fully decouple the dependencies between the categories as you attempted to do.
