title: PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion
    Models
summary: Personalized text-to-image generation has emerged as a powerful and
  sought-after tool, empowering users to create customized images based on their
  specific concepts and prompts. However, existing approaches to personalization
  encounter multiple challenges, including long tuning times, large storage
  requirements, the necessity for multiple input images per identity, and
  limitations in preserving identity and editability. To address these obstacles,
  we present PhotoVerse, an innovative methodology that incorporates a
  dual-branch conditioning mechanism in both text and image domains, providing
  effective control over the image generation process. Furthermore, we introduce
  facial identity loss as a novel component to enhance the preservation of
  identity during training. Remarkably, our proposed PhotoVerse eliminates the
  need for test time tuning and relies solely on a single facial photo of the
  target identity, significantly reducing the resource cost associated with image
  generation. After a single training phase, our approach enables generating
  high-quality images within only a few seconds. Moreover, our method can produce
  diverse images that encompass various scenes and styles. The extensive
  evaluation demonstrates the superior performance of our approach, which
  achieves the dual objectives of preserving identity and facilitating
  editability. Project page: https://photoverse2d.github.io/
paper_authors._id: 650117784be99045d1c0bc5e
paper_authors.name: Li Chen
paper_authors._id: 650117784be99045d1c0bc5f
paper_authors.name: Mengyi Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-13T07:36:06.355Z
paper_authors._id: 6498038ece9190ebb8693034
paper_authors.avatarUrl: /avatars/06ec2457932e05572d917ba286cdef25.svg
paper_authors.fullname: Zhao
paper_authors.type: user
paper_authors.user: Mengyi
paper_authors._id: 650117784be99045d1c0bc60
paper_authors.name: Yiheng Liu
paper_authors._id: 650117784be99045d1c0bc61
paper_authors.name: Mingxu Ding
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-13T07:36:33.134Z
paper_authors._id: 643501b7901c5734bcb58b62
paper_authors.avatarUrl: /avatars/983f34115768be49192a797e32efec3b.svg
paper_authors.fullname: MingxuDing
paper_authors.type: user
paper_authors.user: Dingmx
paper_authors._id: 650117784be99045d1c0bc62
paper_authors.name: Yangyang Song
paper_authors._id: 650117784be99045d1c0bc63
paper_authors.name: Shizun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-13T07:36:48.041Z
paper_authors._id: 63492af03caab0136df08d25
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63492af03caab0136df08d25/-aZzajKl38jT29A-kziCI.jpeg
paper_authors.fullname: Shizun Wang
paper_authors.type: user
paper_authors.user: littlepure2333
paper_authors._id: 650117784be99045d1c0bc64
paper_authors.name: Xu Wang
paper_authors._id: 650117784be99045d1c0bc65
paper_authors.name: Hao Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-13T13:19:17.736Z
paper_authors._id: 6322d55796d6c8518eaff8c5
paper_authors.avatarUrl: /avatars/7309e77145674d01dbd842b0dcda171f.svg
paper_authors.fullname: Hao Yang
paper_authors.type: user
paper_authors.user: hayaalex
paper_authors._id: 650117784be99045d1c0bc66
paper_authors.name: Jing Liu
paper_authors._id: 650117784be99045d1c0bc67
paper_authors.name: Kang Du
paper_authors._id: 650117784be99045d1c0bc68
paper_authors.name: Min Zheng
paper_title: PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion
    Models
paper_summary: Personalized text-to-image generation has emerged as a powerful and
  sought-after tool, empowering users to create customized images based on their
  specific concepts and prompts. However, existing approaches to personalization
  encounter multiple challenges, including long tuning times, large storage
  requirements, the necessity for multiple input images per identity, and
  limitations in preserving identity and editability. To address these obstacles,
  we present PhotoVerse, an innovative methodology that incorporates a
  dual-branch conditioning mechanism in both text and image domains, providing
  effective control over the image generation process. Furthermore, we introduce
  facial identity loss as a novel component to enhance the preservation of
  identity during training. Remarkably, our proposed PhotoVerse eliminates the
  need for test time tuning and relies solely on a single facial photo of the
  target identity, significantly reducing the resource cost associated with image
  generation. After a single training phase, our approach enables generating
  high-quality images within only a few seconds. Moreover, our method can produce
  diverse images that encompass various scenes and styles. The extensive
  evaluation demonstrates the superior performance of our approach, which
  achieves the dual objectives of preserving identity and facilitating
  editability. Project page: https://photoverse2d.github.io/
paper_ai_keywords: dual-branch conditioning mechanism facial identity loss test time tuning high-quality images editability