title: ConceptLab: Creative Generation using Diffusion Prior Constraints
summary: Recent text-to-image generative models have enabled us to transform our words
  into vibrant, captivating imagery. The surge of personalization techniques that
  has followed has also allowed us to imagine unique concepts in new scenes.
  However, an intriguing question remains: How can we generate a new, imaginary
  concept that has never been seen before? In this paper, we present the task of
  creative text-to-image generation, where we seek to generate new members of a
  broad category (e.g., generating a pet that differs from all existing pets). We
  leverage the under-studied Diffusion Prior models and show that the creative
  generation problem can be formulated as an optimization process over the output
  space of the diffusion prior, resulting in a set of "prior constraints". To
  keep our generated concept from converging into existing members, we
  incorporate a question-answering model that adaptively adds new constraints to
  the optimization problem, encouraging the model to discover increasingly more
  unique creations. Finally, we show that our prior constraints can also serve as
  a strong mixing mechanism allowing us to create hybrids between generated
  concepts, introducing even more flexibility into the creative process.
paper_authors._id: 64d1b025e8df1f66ddbcb4fe
paper_authors.name: Elad Richardson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-08T13:57:12.450Z
paper_authors._id: 633f370c13e836a0fc513724
paper_authors.avatarUrl: /avatars/38f3f9ecc5b9f35026becfb0c5c94fdf.svg
paper_authors.fullname: Elad Richardson
paper_authors.type: user
paper_authors.user: eladrich
paper_authors._id: 64d1b025e8df1f66ddbcb4ff
paper_authors.name: Kfir Goldberg
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-08T13:45:11.069Z
paper_authors._id: 6381e8c72a85b6714b7eb5f7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6381e8c72a85b6714b7eb5f7/GTqChXYVI1Q11btXn1X36.jpeg
paper_authors.fullname: kfir goldberg
paper_authors.type: user
paper_authors.user: kfirgold99
paper_authors._id: 64d1b025e8df1f66ddbcb500
paper_authors.name: Yuval Alaluf
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-08T13:57:32.893Z
paper_authors._id: 6310c1e0e87051f3e3e3bcc1
paper_authors.avatarUrl: /avatars/61b24a2eace7da0e27fbc4369a046051.svg
paper_authors.fullname: Yuval Alaluf
paper_authors.type: user
paper_authors.user: yuvalalaluf
paper_authors._id: 64d1b025e8df1f66ddbcb501
paper_authors.name: Daniel Cohen-Or
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-08T13:57:53.900Z
paper_authors._id: 628507161949ebcae8e24ec3
paper_authors.avatarUrl: /avatars/008ecb3daa4c8187b5f339f1176b3c39.svg
paper_authors.fullname: Daniel Cohen-Or
paper_authors.type: user
paper_authors.user: cohenor
paper_title: ConceptLab: Creative Generation using Diffusion Prior Constraints
paper_summary: Recent text-to-image generative models have enabled us to transform our words
  into vibrant, captivating imagery. The surge of personalization techniques that
  has followed has also allowed us to imagine unique concepts in new scenes.
  However, an intriguing question remains: How can we generate a new, imaginary
  concept that has never been seen before? In this paper, we present the task of
  creative text-to-image generation, where we seek to generate new members of a
  broad category (e.g., generating a pet that differs from all existing pets). We
  leverage the under-studied Diffusion Prior models and show that the creative
  generation problem can be formulated as an optimization process over the output
  space of the diffusion prior, resulting in a set of "prior constraints". To
  keep our generated concept from converging into existing members, we
  incorporate a question-answering model that adaptively adds new constraints to
  the optimization problem, encouraging the model to discover increasingly more
  unique creations. Finally, we show that our prior constraints can also serve as
  a strong mixing mechanism allowing us to create hybrids between generated
  concepts, introducing even more flexibility into the creative process.
paper_ai_summary: Creative text-to-image generation leverages Diffusion Prior models to produce unique images by optimizing with adaptive constraints and creating hybrid concepts.
paper_ai_keywords: Diffusion Prior models optimization process prior constraints question-answering model hybrid concepts creative text-to-image generation