Hi everyone,
I had some thoughts on the gaussian prior on VAE latent variables. For instance, Kingma et al. in the original paper ([1312.6114] Auto-Encoding Variational Bayes 2) illustrated the manifold learned on MNIST data by a VAE with 2 latent variables.
Let’s say for the sake of the argument that images whose latent variables are around (0,0) are images of 3. The gaussian prior being centered at zero, 3 images will be more likely to be generated than sixes or tens…
If I understand it right, this is clearly an unwanted side effect of the gaussian priors.
Does anyone have an opinion on this ?
Thanks,
Happy Christmas all,
Matheo