How disentaglement in latent space can produce poor variety of instances in VAE..? I'm reading about $\beta$-VAE which essentially proposes a way to disentangle representations in the latent space. We can subjectively (I guess) identify axes carrying specific sources of variations of (for example) an image and modulate them to produce variety on generated instances.
Shortly speaking what we do is modifying the weight of KL divergence in the loss function, which will be
$$\mathcal{L}(\phi,\theta) = -\mathbb{E}_{\textbf{z}\sim q_\phi(\textbf{z}|\textbf{x})}[\log(p_\theta(\textbf{x}|\textbf{z})] + \beta D_{KL}(q_\phi(\textbf{z}|\textbf{x}) || p(\textbf{z}))$$
I'm trying to understand more what is happening here, if $\beta >1$ then we are posing more attention on minimizing the KL divergence, therefore we are enforcing our posterior to look closer to a normal distribution with zero mean and unit variance... how is this connected to disentaglement?
Also, it has recently happened to me some sort of inverse situation (that I've noticed is pretty common): on trained (vanilla) VAEs for some of my projects, I observed that generated samples looked quite all the same, and I have partially resolved the issue by imposing a very small value of $\beta$ such as $0.0001$.
In this alternative case, how the disentaglement is related to the poor variability in generation?

I respond by quoting elements from the two references on $\beta$-VAE, that you should read for much more details if you haven't.
Disentanglement as defined in Understanding disentangling in $\beta$-VAE by C. Burgess is defined as

A disentangled representation can be defined as one where single latent units are sensitive to changes
in single generative factors, while being relatively invariant to changes in other factors.

To tend towards this behaviour, conclusions that higher values of $\beta$ are required are made in $\beta$-VAE: Learning basic visual concepts with constrained variational framework by I. Higgins.

These constraints limit the capacity of $z$, which, combined with the pressure to maximise the log likelihood of the training data x under the model, should encourage the model to learn the most efficient representation of the data.

Finally, it is hard to say what is happening in your own experiment. Perhaps you could investigate a disentanglement metric as proposed in the second article, or maybe you just had a scaling problem between your two terms in your ELBO....
