title: Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and
    Latent Diffusion
summary: Text-to-image generation is a significant domain in modern computer vision
  and has achieved substantial improvements through the evolution of generative
  architectures. Among these, there are diffusion-based models that have
  demonstrated essential quality enhancements. These models are generally split
  into two categories: pixel-level and latent-level approaches. We present
  Kandinsky1, a novel exploration of latent diffusion architecture, combining the
  principles of the image prior models with latent diffusion techniques. The
  image prior model is trained separately to map text embeddings to image
  embeddings of CLIP. Another distinct feature of the proposed model is the
  modified MoVQ implementation, which serves as the image autoencoder component.
  Overall, the designed model contains 3.3B parameters. We also deployed a
  user-friendly demo system that supports diverse generative modes such as
  text-to-image generation, image fusion, text and image fusion, image variations
  generation, and text-guided inpainting/outpainting. Additionally, we released
  the source code and checkpoints for the Kandinsky models. Experimental
  evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking
  our model as the top open-source performer in terms of measurable image
  generation quality.
paper_authors._id: 651f6455be34a5f2cf5d9fbe
paper_authors.name: Anton Razzhigaev
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-09T07:51:17.236Z
paper_authors._id: 6172aaeec8e66e2aa84c06b9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6172aaeec8e66e2aa84c06b9/ZdRZSp3P1SU6CIDbvQwkv.jpeg
paper_authors.fullname: Anton Razzhigaev
paper_authors.type: user
paper_authors.user: razzant
paper_authors._id: 651f6455be34a5f2cf5d9fbf
paper_authors.name: Arseniy Shakhmatov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-10T13:24:16.263Z
paper_authors._id: 62850e7c5ade86339a213802
paper_authors.avatarUrl: /avatars/3ed13812c1f0ef938071c6fdf208b3f8.svg
paper_authors.fullname: Arseniy Shahmatov
paper_authors.type: user
paper_authors.user: Cene655
paper_authors._id: 651f6455be34a5f2cf5d9fc0
paper_authors.name: Anastasia Maltseva
paper_authors._id: 651f6455be34a5f2cf5d9fc1
paper_authors.name: Vladimir Arkhipkin
paper_authors._id: 651f6455be34a5f2cf5d9fc2
paper_authors.name: Igor Pavlov
paper_authors._id: 651f6455be34a5f2cf5d9fc3
paper_authors.name: Ilya Ryabov
paper_authors._id: 651f6455be34a5f2cf5d9fc4
paper_authors.name: Angelina Kuts
paper_authors._id: 651f6455be34a5f2cf5d9fc5
paper_authors.name: Alexander Panchenko
paper_authors._id: 651f6455be34a5f2cf5d9fc6
paper_authors.name: Andrey Kuznetsov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-09T07:50:48.965Z
paper_authors._id: 643984dceb7c5616ef3f5d54
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643984dceb7c5616ef3f5d54/10JRkblrRIEVci6UJwvPz.jpeg
paper_authors.fullname: Andrey Kuznetsov
paper_authors.type: user
paper_authors.user: kuznetsoffandrey
paper_authors._id: 651f6455be34a5f2cf5d9fc7
paper_authors.name: Denis Dimitrov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-28T13:07:33.552Z
paper_authors._id: 6669a678465d1d802181e456
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6669a678465d1d802181e456/ZCthBBhDFQnh0bBkgUQUU.png
paper_authors.fullname: Denis Dimitrov
paper_authors.type: user
paper_authors.user: dendimitrov
paper_title: Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and
    Latent Diffusion
paper_summary: Text-to-image generation is a significant domain in modern computer vision
  and has achieved substantial improvements through the evolution of generative
  architectures. Among these, there are diffusion-based models that have
  demonstrated essential quality enhancements. These models are generally split
  into two categories: pixel-level and latent-level approaches. We present
  Kandinsky1, a novel exploration of latent diffusion architecture, combining the
  principles of the image prior models with latent diffusion techniques. The
  image prior model is trained separately to map text embeddings to image
  embeddings of CLIP. Another distinct feature of the proposed model is the
  modified MoVQ implementation, which serves as the image autoencoder component.
  Overall, the designed model contains 3.3B parameters. We also deployed a
  user-friendly demo system that supports diverse generative modes such as
  text-to-image generation, image fusion, text and image fusion, image variations
  generation, and text-guided inpainting/outpainting. Additionally, we released
  the source code and checkpoints for the Kandinsky models. Experimental
  evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking
  our model as the top open-source performer in terms of measurable image
  generation quality.
paper_ai_keywords: diffusion-based models pixel-level approaches latent-level approaches latent diffusion architecture image prior models text embeddings image embeddings CLIP MoVQ (Modified Vector Quantization) image autoencoder FID (Fréchet Inception Distance) COCO-30K dataset text-to-image generation image fusion text and image fusion image variations generation text-guided inpainting/outpainting