title: Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
summary: We present Face0, a novel way to instantaneously condition a text-to-image
  generation model on a face, in sample time, without any optimization procedures
  such as fine-tuning or inversions. We augment a dataset of annotated images
  with embeddings of the included faces and train an image generation model, on
  the augmented dataset. Once trained, our system is practically identical at
  inference time to the underlying base model, and is therefore able to generate
  images, given a user-supplied face image and a prompt, in just a couple of
  seconds. Our method achieves pleasing results, is remarkably simple, extremely
  fast, and equips the underlying model with new capabilities, like controlling
  the generated images both via text or via direct manipulation of the input face
  embeddings. In addition, when using a fixed random vector instead of a face
  embedding from a user supplied image, our method essentially solves the problem
  of consistent character generation across images. Finally, while requiring
  further research, we hope that our method, which decouples the model's textual
  biases from its biases on faces, might be a step towards some mitigation of
  biases in future text-to-image models.
paper_authors._id: 6487d5275d164d8c7f8ed0bf
paper_authors.name: Dani Valevski
paper_authors._id: 6487d5275d164d8c7f8ed0c0
paper_authors.name: Danny Wasserman
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-13T14:22:29.242Z
paper_authors._id: 6488751b7e6b887ab77ce30d
paper_authors.avatarUrl: /avatars/7fb9ef379f22d0190daab1e5ac548bc4.svg
paper_authors.fullname: Danny Lumen
paper_authors.type: user
paper_authors.user: DannyWasserman
paper_authors._id: 6487d5275d164d8c7f8ed0c1
paper_authors.name: Yossi Matias
paper_authors._id: 6487d5275d164d8c7f8ed0c2
paper_authors.name: Yaniv Leviathan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-13T17:39:08.336Z
paper_authors._id: 6394a2244db1a9bedd1f2229
paper_authors.avatarUrl: /avatars/f04496811eb6873781e2ba606a2f97f0.svg
paper_authors.fullname: Yaniv Leviathan
paper_authors.type: user
paper_authors.user: yanivle
paper_title: Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
paper_summary: We present Face0, a novel way to instantaneously condition a text-to-image
  generation model on a face, in sample time, without any optimization procedures
  such as fine-tuning or inversions. We augment a dataset of annotated images
  with embeddings of the included faces and train an image generation model, on
  the augmented dataset. Once trained, our system is practically identical at
  inference time to the underlying base model, and is therefore able to generate
  images, given a user-supplied face image and a prompt, in just a couple of
  seconds. Our method achieves pleasing results, is remarkably simple, extremely
  fast, and equips the underlying model with new capabilities, like controlling
  the generated images both via text or via direct manipulation of the input face
  embeddings. In addition, when using a fixed random vector instead of a face
  embedding from a user supplied image, our method essentially solves the problem
  of consistent character generation across images. Finally, while requiring
  further research, we hope that our method, which decouples the model's textual
  biases from its biases on faces, might be a step towards some mitigation of
  biases in future text-to-image models.
paper_ai_keywords: text-to-image generation embeddings image generation model face embedding input face embeddings consistent character generation