Boosting Generative Image Modeling via Joint Image-Feature Synthesis
Latent diffusion models (LDMs) dominate high-quality image generation, yet
integrating representation learning with generative modeling remains a
challenge. We introduce a novel generative image modeling framework that
seamlessly bridges this gap by leveraging a diffusion model to jointly model
low-level image latents (from a variational autoencoder) and high-level
semantic features (from a pretrained self-supervised encoder like DINO). Our
latent-semantic diffusion approach learns to generate coherent image-feature
pairs from pure noise, significantly enhancing both generative quality and
training efficiency, all while requiring only minimal modifications to standard
Diffusion Transformer architectures. By eliminating the need for complex
distillation objectives, our unified design simplifies training and unlocks a
powerful new inference strategy: Representation Guidance, which leverages
learned semantics to steer and refine image generation. Evaluated in both
conditional and unconditional settings, our method delivers substantial
improvements in image quality and training convergence speed, establishing a
new direction for representation-aware generative modeling.
680b494c6bd146aa35ab2e1c
Theodoros Kouzelis
680b494c6bd146aa35ab2e1d
Efstathios Karypidis
claimed_verified
2025-04-25T12:27:13.393Z
677272184d148b904333e874
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/5dUau7gxLk4Wm1TiiJJri.jpeg
Efstathios Karypidis
user
Sta8is
680b494c6bd146aa35ab2e1e
Ioannis Kakogeorgiou
claimed_verified
2025-04-25T12:27:11.569Z
661ba524bd9243bf7e598355
https://cdn-avatars.huggingface.co/v1/production/uploads/661ba524bd9243bf7e598355/i77yD4XgJn2vUbn_mIsT8.jpeg
Ioannis Kakogeorgiou
user
gkakogeorgiou
680b494c6bd146aa35ab2e1f
Spyros Gidaris
claimed_verified
2025-04-28T13:50:08.450Z
678cd947d13c19c917644792
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/010zyg-JoIjaqVn9lSDRT.png
Spyros Gidaris
user
sgidaris
680b494c6bd146aa35ab2e20
Nikos Komodakis
Boosting Generative Image Modeling via Joint Image-Feature Synthesis
Latent diffusion models (LDMs) dominate high-quality image generation, yet
integrating representation learning with generative modeling remains a
challenge. We introduce a novel generative image modeling framework that
seamlessly bridges this gap by leveraging a diffusion model to jointly model
low-level image latents (from a variational autoencoder) and high-level
semantic features (from a pretrained self-supervised encoder like DINO). Our
latent-semantic diffusion approach learns to generate coherent image-feature
pairs from pure noise, significantly enhancing both generative quality and
training efficiency, all while requiring only minimal modifications to standard
Diffusion Transformer architectures. By eliminating the need for complex
distillation objectives, our unified design simplifies training and unlocks a
powerful new inference strategy: Representation Guidance, which leverages
learned semantics to steer and refine image generation. Evaluated in both
conditional and unconditional settings, our method delivers substantial
improvements in image quality and training convergence speed, establishing a
new direction for representation-aware generative modeling.
A novel generative image modeling framework uses latent-semantic diffusion to learn coherent image-feature pairs, enhancing generative quality and training efficiency with minimal modifications to standard architectures.
latent diffusion models variational autoencoder pretrained self-supervised encoder DINO latent-semantic diffusion generative quality training efficiency Diffusion Transformer Representation Guidance image quality training convergence speed