Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and
  Latent Diffusion
Text-to-image generation is a significant domain in modern computer vision
and has achieved substantial improvements through the evolution of generative
architectures. Among these, there are diffusion-based models that have
demonstrated essential quality enhancements. These models are generally split
into two categories: pixel-level and latent-level approaches. We present
Kandinsky1, a novel exploration of latent diffusion architecture, combining the
principles of the image prior models with latent diffusion techniques. The
image prior model is trained separately to map text embeddings to image
embeddings of CLIP. Another distinct feature of the proposed model is the
modified MoVQ implementation, which serves as the image autoencoder component.
Overall, the designed model contains 3.3B parameters. We also deployed a
user-friendly demo system that supports diverse generative modes such as
text-to-image generation, image fusion, text and image fusion, image variations
generation, and text-guided inpainting/outpainting. Additionally, we released
the source code and checkpoints for the Kandinsky models. Experimental
evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking
our model as the top open-source performer in terms of measurable image
generation quality.
651f6455be34a5f2cf5d9fbe
Anton Razzhigaev
claimed_verified
2023-10-09T07:51:17.236Z
6172aaeec8e66e2aa84c06b9
https://cdn-avatars.huggingface.co/v1/production/uploads/6172aaeec8e66e2aa84c06b9/ZdRZSp3P1SU6CIDbvQwkv.jpeg
Anton Razzhigaev
user
razzant
651f6455be34a5f2cf5d9fbf
Arseniy Shakhmatov
claimed_verified
2023-10-10T13:24:16.263Z
62850e7c5ade86339a213802
/avatars/3ed13812c1f0ef938071c6fdf208b3f8.svg
Arseniy Shahmatov
user
Cene655
651f6455be34a5f2cf5d9fc0
Anastasia Maltseva
651f6455be34a5f2cf5d9fc1
Vladimir Arkhipkin
651f6455be34a5f2cf5d9fc2
Igor Pavlov
651f6455be34a5f2cf5d9fc3
Ilya Ryabov
651f6455be34a5f2cf5d9fc4
Angelina Kuts
651f6455be34a5f2cf5d9fc5
Alexander Panchenko
651f6455be34a5f2cf5d9fc6
Andrey Kuznetsov
claimed_verified
2023-10-09T07:50:48.965Z
643984dceb7c5616ef3f5d54
https://cdn-avatars.huggingface.co/v1/production/uploads/643984dceb7c5616ef3f5d54/10JRkblrRIEVci6UJwvPz.jpeg
Andrey Kuznetsov
user
kuznetsoffandrey
651f6455be34a5f2cf5d9fc7
Denis Dimitrov
claimed_verified
2024-06-28T13:07:33.552Z
6669a678465d1d802181e456
https://cdn-avatars.huggingface.co/v1/production/uploads/6669a678465d1d802181e456/ZCthBBhDFQnh0bBkgUQUU.png
Denis Dimitrov
user
dendimitrov
Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and
  Latent Diffusion
Text-to-image generation is a significant domain in modern computer vision
and has achieved substantial improvements through the evolution of generative
architectures. Among these, there are diffusion-based models that have
demonstrated essential quality enhancements. These models are generally split
into two categories: pixel-level and latent-level approaches. We present
Kandinsky1, a novel exploration of latent diffusion architecture, combining the
principles of the image prior models with latent diffusion techniques. The
image prior model is trained separately to map text embeddings to image
embeddings of CLIP. Another distinct feature of the proposed model is the
modified MoVQ implementation, which serves as the image autoencoder component.
Overall, the designed model contains 3.3B parameters. We also deployed a
user-friendly demo system that supports diverse generative modes such as
text-to-image generation, image fusion, text and image fusion, image variations
generation, and text-guided inpainting/outpainting. Additionally, we released
the source code and checkpoints for the Kandinsky models. Experimental
evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking
our model as the top open-source performer in terms of measurable image
generation quality.
Kandinsky1, a latent diffusion architecture, achieves high-quality text-to-image generation by integrating image prior models and modified MoVQ autoencoders, outperforming other open-source models.
diffusion-based models pixel-level latent-level image prior models latent diffusion techniques CLIP MoVQ image autoencoder text-to-image generation image fusion text and image fusion image variations generation text-guided inpainting/outpainting FID score