Hi @googlebot  thanks for the reply although it seems (from some papers and from this topic: machine learning - When generating samples using variational autoencoder, we decode samples from $N(0,1)$ instead of $\mu + \sigma N(0,1)$ - Cross Validated 1) that sampling from the prior is the usual way.
I have another interrogation that is related to the first one: I have trained a conditional 2-latent variables VAE on MNIST dataset. The generated images are very convincing.  When plotting the latent representations of the generated samples (see figure below), I expected to see clusters which is not the case at all. This is actually what you said :

So, yeah, posterior clusters are scattered around zero randomly, but encoder deals with that.

Each color is related to a different number. X and Y axis are the two dimensional latent variables.
I’m quite puzzled, there’s obviously something I don’t get… I thought that small variations of latent variables would induce small variations in generated images as it is presented in Kingma paper on MNIST and Frey Face datasets.
latent_variables_mnist1848×977 29.4 KB
What do you mean by :

note that paper you linked uses quantiles for that plot

The paper says:

since the prior of the latent space is Gaussian, linearly spaced coor-dinates on the unit square were transformed through the inverse CDF of the Gaussian to produce values of the latent variablesz. For each of these values z, we plotted the corresponding generative pθ(x|z)with the learned parameters θ

which I don’t understand.