VAE : How is likelihood $p(x|z)$ defined? Disclaimer : not a strong background in Bayesian statistics.
I gather from questions such as this one and this one that in the context of VAEs, we suppose that we know the (form of the ?) prior $p(z)$ and the likelihood $p(x|z)$. What we don't know is the posterior $p(z|x)$ and the marginal $p(x)$, the whole point of the VAE training process being to approximate them. However, I don't understand how we define the likelihood $p(x|z)$ and what form does it usually take.
Intuitively, I'd think that we would set it as some Gaussian for it be easily conjugable with the prior, to facilitate the computation of the posterior. Indeed, Wikipedia seems to say that $p(x|z)$ is usually defined as a Normal distribution :


*

*First, define a simple distribution $p(z)$ over a latent random variable $Z$. Usually a normal distribution or a uniform distribution
suffices.

*Next, define a family of complicated functions $f_{\theta }$ (such as a deep neural network) parametrized by $\theta$ .

*Finally, define a way to convert any $f_{\theta }(z)$ into a simple distribution over the observable random variable $X$. For example, let
$f_{\theta }(z)=(f_{1}(z),f_{2}(z))$ have two outputs, then we can
define the corresponding distribution over $X$ to be the normal
distribution $N(f_{1}(z),e^{f_{2}(z)})$.


But I thought that this was this "implictly parametrized distribution" "trick" was how we set $p(z|x)$ : As shown in the image below, it is $z$ which is sampled from a Normal distribution parametrized by $x$. In this image, $p(x|z)$ is defined as the function learned by a neural network, which is certainly not a "simple" function.

(image is from here)

It depends on your output domain

*

*It can be a Multivariate Gaussian
$$p(x|z) \sim \mathcal{N}(z, \mu_\phi(z),\Sigma_\phi(z))$$

*For images, you can also take a categorical distribution over the pixel values.
$$p(x|z) \sim \text{Categorical}(x|\phi(z))$$
In practice, this translates to a neural network where you take the softmax over the pixel values.

You can then learn the parameterization $\phi$ of $p_\phi(x|z)$ by MLE, i.e. maximizing
$$\mathbb{E}_{z \sim q_\theta(z)}[\log p_\phi(x|z)]$$

Indeed, Wikipedia seems to say that p(x|z)
is usually defined as a Normal distribution

I think Wikipedia is talking about the parameterization of $q(z|x)$ in the passage you cited. $f_1(z)$ computes $\mu_\theta$ and $f_2(z)$ computes $\log \Sigma_\theta$.
This is the distribution from where you sample $z \sim \mathcal{N}(z,\mu_\phi,\Sigma^2_\phi)$, which you then feed into the decoder.
You can find a detailed description of variational autoencoders here
