EQ-VAE: Equivariance Regularized Latent Space for Improved Generative
  Image Modeling
Latent generative models have emerged as a leading approach for high-quality
image synthesis. These models rely on an autoencoder to compress images into a
latent space, followed by a generative model to learn the latent distribution.
We identify that existing autoencoders lack equivariance to semantic-preserving
transformations like scaling and rotation, resulting in complex latent spaces
that hinder generative performance. To address this, we propose EQ-VAE, a
simple regularization approach that enforces equivariance in the latent space,
reducing its complexity without degrading reconstruction quality. By finetuning
pre-trained autoencoders with EQ-VAE, we enhance the performance of several
state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT,
achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning.
EQ-VAE is compatible with both continuous and discrete autoencoders, thus
offering a versatile enhancement for a wide range of latent generative models.
Project page and code: https://eq-vae.github.io/.
67b4e4259beded220ad14729
Theodoros Kouzelis
claimed_verified
2025-02-21T16:15:22.578Z
6383aa17834d3558a3955186
/avatars/1f6aed0a762379df334bc6a734d42f86.svg
Kouzelis
user
zelaki
67b4e4259beded220ad1472a
Ioannis Kakogeorgiou
claimed_verified
2025-02-19T09:03:49.086Z
661ba524bd9243bf7e598355
https://cdn-avatars.huggingface.co/v1/production/uploads/661ba524bd9243bf7e598355/i77yD4XgJn2vUbn_mIsT8.jpeg
Ioannis Kakogeorgiou
user
gkakogeorgiou
67b4e4259beded220ad1472b
Spyros Gidaris
claimed_verified
2025-04-28T13:50:15.651Z
678cd947d13c19c917644792
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/010zyg-JoIjaqVn9lSDRT.png
Spyros Gidaris
user
sgidaris
67b4e4259beded220ad1472c
Nikos Komodakis
EQ-VAE: Equivariance Regularized Latent Space for Improved Generative
  Image Modeling
Latent generative models have emerged as a leading approach for high-quality
image synthesis. These models rely on an autoencoder to compress images into a
latent space, followed by a generative model to learn the latent distribution.
We identify that existing autoencoders lack equivariance to semantic-preserving
transformations like scaling and rotation, resulting in complex latent spaces
that hinder generative performance. To address this, we propose EQ-VAE, a
simple regularization approach that enforces equivariance in the latent space,
reducing its complexity without degrading reconstruction quality. By finetuning
pre-trained autoencoders with EQ-VAE, we enhance the performance of several
state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT,
achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning.
EQ-VAE is compatible with both continuous and discrete autoencoders, thus
offering a versatile enhancement for a wide range of latent generative models.
Project page and code: https://eq-vae.github.io/.
EQ-VAE enhances latent generative models by enforcing equivariance in the latent space, improving performance without degrading reconstruction quality.
latent generative models autoencoder latent space equivariance semantic-preserving transformations scaling rotation generative performance EQ-VAE regularization continuous autoencoders discrete autoencoders DiT SiT REPA MaskGIT SD-VAE fine-tuning