title: Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image
    Models
summary: Text-to-image (T2I) personalization allows users to guide the creative image
  generation process by combining their own visual concepts in natural language
  prompts. Recently, encoder-based techniques have emerged as a new effective
  approach for T2I personalization, reducing the need for multiple images and
  long training times. However, most existing encoders are limited to a
  single-class domain, which hinders their ability to handle diverse concepts. In
  this work, we propose a domain-agnostic method that does not require any
  specialized dataset or prior information about the personalized concepts. We
  introduce a novel contrastive-based regularization technique to maintain high
  fidelity to the target concept characteristics while keeping the predicted
  embeddings close to editable regions of the latent space, by pushing the
  predicted tokens toward their nearest existing CLIP tokens. Our experimental
  results demonstrate the effectiveness of our approach and show how the learned
  tokens are more semantic than tokens predicted by unregularized models. This
  leads to a better representation that achieves state-of-the-art performance
  while being more flexible than previous methods.
paper_authors._id: 64b0a64135552c1cc148f1cb
paper_authors.name: Moab Arar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T11:25:11.962Z
paper_authors._id: 60defb17bb3e1aaa18b005bb
paper_authors.avatarUrl: /avatars/086264e93d6f7ba2eab5cc1b218df4a5.svg
paper_authors.fullname: Moab Arar
paper_authors.type: user
paper_authors.user: moabarar
paper_authors._id: 64b0a64135552c1cc148f1cc
paper_authors.name: Rinon Gal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T11:25:20.012Z
paper_authors._id: 627c1360f19c5eb46d55ba05
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652712871747-627c1360f19c5eb46d55ba05.jpeg
paper_authors.fullname: Rinon Gal
paper_authors.type: user
paper_authors.user: rinong
paper_authors._id: 64b0a64135552c1cc148f1cd
paper_authors.name: Yuval Atzmon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T11:25:28.210Z
paper_authors._id: 6310627fa84f681a1193b372
paper_authors.avatarUrl: /avatars/53ac50eb70f8fcf8c255e9bc0207790a.svg
paper_authors.fullname: Yuval Atzmon
paper_authors.type: user
paper_authors.user: yatzmon
paper_authors._id: 64b0a64135552c1cc148f1ce
paper_authors.name: Gal Chechik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T11:25:37.759Z
paper_authors._id: 6493393f357b252af72196c5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
paper_authors.fullname: Gal Chechik
paper_authors.type: user
paper_authors.user: galchechik
paper_authors._id: 64b0a64135552c1cc148f1cf
paper_authors.name: Daniel Cohen-Or
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T11:25:49.297Z
paper_authors._id: 628507161949ebcae8e24ec3
paper_authors.avatarUrl: /avatars/008ecb3daa4c8187b5f339f1176b3c39.svg
paper_authors.fullname: Daniel Cohen-Or
paper_authors.type: user
paper_authors.user: cohenor
paper_authors._id: 64b0a64135552c1cc148f1d0
paper_authors.name: Ariel Shamir
paper_authors._id: 64b0a64135552c1cc148f1d1
paper_authors.name: Amit H. Bermano
paper_title: Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image
    Models
paper_summary: Text-to-image (T2I) personalization allows users to guide the creative image
  generation process by combining their own visual concepts in natural language
  prompts. Recently, encoder-based techniques have emerged as a new effective
  approach for T2I personalization, reducing the need for multiple images and
  long training times. However, most existing encoders are limited to a
  single-class domain, which hinders their ability to handle diverse concepts. In
  this work, we propose a domain-agnostic method that does not require any
  specialized dataset or prior information about the personalized concepts. We
  introduce a novel contrastive-based regularization technique to maintain high
  fidelity to the target concept characteristics while keeping the predicted
  embeddings close to editable regions of the latent space, by pushing the
  predicted tokens toward their nearest existing CLIP tokens. Our experimental
  results demonstrate the effectiveness of our approach and show how the learned
  tokens are more semantic than tokens predicted by unregularized models. This
  leads to a better representation that achieves state-of-the-art performance
  while being more flexible than previous methods.
paper_ai_keywords: contrastive-based regularization fidelity target concept characteristics latent space predicted tokens CLIP tokens semantic representation state-of-the-art performance