Clarification on normal distribution assumption for probablistic decoder P(x|z) in VAE After searching around on the internet, it seems that the following assumption seems to be commonly used in VAEs:

*

*For a continuous domain (MNIST), assume $p(x|z) = N(x; f(z), \sigma^2)$. Where $f(z)$ is the output of the decoder for a specific pixel value. In practice, we set the variance to some handpicked value. Not sure why.


*For a discrete (MNIST, but binarized), assume $P(x|z) = Bernoulli(x; \theta = f(z))$, where $f(z)$ is the decoder output as before, but specifically passed through a sigmoid to specify a probability.
The Bernoulli assumption sort of makes sense, since the Bernoulli distribution seems to fit the domain of binarized pixels.
However, for the continuous case, why do we assume that $P(x|z)$ is normally distributed? When does this assumption break? Why do we set the variance to some constant rather than having an auxiliary decoder on the latent code for the variance?

The main reason for presuming the decoder to be Gaussian is because that makes everything easier without too much constraining your solutions. I have checked the original paper, and they do not presume the variance from the decoder to be constant. So an additional auxiliary decoder for the variance is not necessary.
