ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders
<|reference_start|>ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders: Traditional Variational Autoencoders (VAEs) are constrained by the limitations of the Evidence Lower Bound (ELBO) formulation, particularly when utilizing simplistic, non-analytic, or unknown prior distributions. These limitations inhibit the VAE's ability to generate high-quality samples and provide clear, interpretable latent representations. This work introduces the Entropy Decomposed Variational Autoencoder (ED-VAE), a novel re-formulation of the ELBO that explicitly includes entropy and cross-entropy components. This reformulation significantly enhances model flexibility, allowing for the integration of complex and non-standard priors. By providing more detailed control over the encoding and regularization of latent spaces, ED-VAE not only improves interpretability but also effectively captures the complex interactions between latent variables and observed data, thus leading to better generative performance.<|reference_end|>
@article{lygerakis2024ed-vae:,
    title={ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders},
    author={Fotios Lygerakis, Elmar Rueckert},
    journal={arXiv preprint arXiv:2407.06797},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.06797},
    primaryClass={cs.LG cs.AI stat.ML}
}