Calculating acceptance probability in Metropolis-Hastings algorithm In the Metropolis-Hastings algorithm, acceptance probability is given as
$$
\alpha = \min \left( 1,\frac{f(\theta^{'}|y)q(\theta|\theta^{'})}{f(\theta|y)q(\theta^{'}|\theta)} \right)
$$
which simplifies to
$$
\alpha = \min \left( 1,\frac{f(y|\theta^{'})f(\theta^{'}) q(\theta|\theta^{'})}{f(y|\theta)f(\theta) q(\theta^{'}|\theta)} \right)
$$
My questions are:

*

*Is it possible to evaluate $f(y|\theta^{'})$ and $f(y|\theta)$ without knowing its analytical form?

*If not, does that mean that we need to know the analytical form of the posterior and the likelihood to use the Metropolis-Hastings algorithm? Also, what is the purpose of the data then?

*If yes, how do we do it?

Thank you in advance!

*

*It is technically possible, but usually unnecessary. It is usually far easier to calculate the likelihood and the prior than it is the posterior.


*No, you do not need to know the posterior a priori. The reason we use Metropolis Hastings is because the full expression for the posterior has a normalizing constant that is difficult to calculate. BUT, you do know it is a constant, that it will make the probabilities sum to 1, and that the posterior is proportional to the likelihood times the prior. If you sample the distribution proportional to the likelihood times the prior, then you can just divide by the total number of samples to get the posterior density instead. The data's purpose is as a test: if the data fit a particular model better, that model will be more likely to be correct.


*Look up "Bayesian Statistical Likelihood", which essentially works by simulating the possible experimental outcomes based on the current model, and constructing a likelihood function around the simulations. That said, this is usually overkill.
