How can Metropolis-Hastings use the function it is trying to approximate? The MH algoithm is used to obtain samples from a probability distribution $f$ that is difficult to sample from directly. The process as described in this answer is:


*

*Pick a initial random state $x_0$.

*Generate a candidate $x^{\star}$ from $Q(\cdot\vert x_0)$.

*Calculate the ratio $\alpha=f(x^{\star})/f(x_0)$.

*Accept $x^{\star}$ as a realisation of $f$ with probability $\alpha$.

*Take $x^{\star}$ as the new initial state and continue sampling until you get the desired sample size.


But how can one compute $\alpha=f(x^{\star})/f(x_0)$ if we are trying to estimate $f$ in the first place? If we are able to calculate this ratio, why do we need to use Metropolis-Hastings?
A similar question was asked here and it linked to this tutorial where the author shows an example of sampling from an exponential distribution, but they use that exact exponential distribution in the sampling process, hence my confusion. Why can't we sample from the distribution directly if it is exactly known?

The object of Metropolis-Hastings is to generate samples from $f$. The function $f$ could be complicated and need numerical evaluation. Metropolis-Hastings addresses the question:
How can I sample from $f$?
But, why would we need samples from $f$ if it is possible to evaluate the function? One example is Monte Carlo integration:
$$
E[h(x)] = \int h(x) f(x)\,dx \underset{LLN}{\approx} \frac{1}{N}\sum h(X_i), \quad\quad X_i \sim f
$$
As an example, consider $f = \frac{1}{c}(x_1^8 + x_2^2)$ and an expected value $E[\cos(x_1) \cdot \cos(x_2)]$. Given a sample, it is very easy to evaluate the expected value. Also, there is no need to calculate $c$ which can be intractable.
