How disentaglement in latent space can produce poor variety of instances in VAE..? I'm reading about $\beta$-VAE which essentially proposes a way to disentangle representations in the latent space. We can subjectively (I guess) identify axes carrying specific sources of variations of (for example) an image and modulate them to produce variety on generated instances.
Shortly speaking what we do is modifying the weight of KL divergence in the loss function, which will be
$$\mathcal{L}(\phi,\theta) = -\mathbb{E}_{\textbf{z}\sim q_\phi(\textbf{z}|\textbf{x})}[\log(p_\theta(\textbf{x}|\textbf{z})] + \beta D_{KL}(q_\phi(\textbf{z}|\textbf{x}) || p(\textbf{z}))$$
I'm trying to understand more what is happening here, if $\beta >1$ then we are posing more attention on minimizing the KL divergence, therefore we are enforcing our posterior to look closer to a normal distribution with zero mean and unit variance... how is this connected to disentaglement?
Also, it has recently happened to me some sort of inverse situation (that I've noticed is pretty common): on trained (vanilla) VAEs for some of my projects, I observed that generated samples looked quite all the same, and I have partially resolved the issue by imposing a very small value of $\beta$ such as $0.0001$.
In this alternative case, how the disentaglement is related to the poor variability in generation?

If you are using beta-VAE for image reconstruction, then probably the reason is that you are calculating the average of loss of all the pixels for the reconstruction loss. This is correct for deterministic autoencoders but not correct for VAEs. For VAE you should calculate the sum of the loss of all the pixels for each image, then calculate the mean of loss of all images in a training batch.
The reason is that in VAE the reconstruction loss is the log probability of generating the data given the latent variables, so for image data the log probability of generating the image is the sum of log probability of generating each individual pixel. If you calculate the mean of all pixels, the resonstruction loss is scaled down by a factor of the number of pixels in the image, and therefore you need much lower beta values for the training to work.
