title: StableIdentity: Inserting Anybody into Anywhere at First Sight
summary: Recent advances in large pretrained text-to-image models have shown
  unprecedented capabilities for high-quality human-centric generation, however,
  customizing face identity is still an intractable problem. Existing methods
  cannot ensure stable identity preservation and flexible editability, even with
  several images for each subject during training. In this work, we propose
  StableIdentity, which allows identity-consistent recontextualization with just
  one face image. More specifically, we employ a face encoder with an identity
  prior to encode the input face, and then land the face representation into a
  space with an editable prior, which is constructed from celeb names. By
  incorporating identity prior and editability prior, the learned identity can be
  injected anywhere with various contexts. In addition, we design a masked
  two-phase diffusion loss to boost the pixel-level perception of the input face
  and maintain the diversity of generation. Extensive experiments demonstrate our
  method outperforms previous customization methods. In addition, the learned
  identity can be flexibly combined with the off-the-shelf modules such as
  ControlNet. Notably, to the best knowledge, we are the first to directly inject
  the identity learned from a single image into video/3D generation without
  finetuning. We believe that the proposed StableIdentity is an important step to
  unify image, video, and 3D customized generation models.
paper_authors._id: 65b88335e0bde92c176880cc
paper_authors.name: Qinghe Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-30T12:43:54.809Z
paper_authors._id: 646f3418a6a58aa29505fd30
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646f3418a6a58aa29505fd30/1z13rnpb6rsUgQsYumWPg.png
paper_authors.fullname: QINGHE WANG
paper_authors.type: user
paper_authors.user: DecoderWQH666
paper_authors._id: 65b88335e0bde92c176880cd
paper_authors.name: Xu Jia
paper_authors._id: 65b88335e0bde92c176880ce
paper_authors.name: Xiaomin Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-30T13:18:03.468Z
paper_authors._id: 6423bfecadbaa37d50c3d9bc
paper_authors.avatarUrl: /avatars/207f26c2d59d990f5740d620c3a399a5.svg
paper_authors.fullname: Xiaomin
paper_authors.type: user
paper_authors.user: XiaominDLUT
paper_authors._id: 65b88335e0bde92c176880cf
paper_authors.name: Taiqing Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-30T12:43:52.319Z
paper_authors._id: 64706f4122d12f52949b0117
paper_authors.avatarUrl: /avatars/c6c7e89fbe1bbbf40e3871d5e4cfc3b4.svg
paper_authors.fullname: LITAIQING
paper_authors.type: user
paper_authors.user: LiTaiQing
paper_authors._id: 65b88335e0bde92c176880d0
paper_authors.name: Liqian Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-30T13:53:47.601Z
paper_authors._id: 63075c0768bd6e30ddac805d
paper_authors.avatarUrl: /avatars/7408b625508f31129f4e68fd38daeba3.svg
paper_authors.fullname: CharlieMa
paper_authors.type: user
paper_authors.user: CharlieMa
paper_authors._id: 65b88335e0bde92c176880d1
paper_authors.name: Yunzhi Zhuge
paper_authors._id: 65b88335e0bde92c176880d2
paper_authors.name: Huchuan Lu
paper_title: StableIdentity: Inserting Anybody into Anywhere at First Sight
paper_summary: Recent advances in large pretrained text-to-image models have shown
  unprecedented capabilities for high-quality human-centric generation, however,
  customizing face identity is still an intractable problem. Existing methods
  cannot ensure stable identity preservation and flexible editability, even with
  several images for each subject during training. In this work, we propose
  StableIdentity, which allows identity-consistent recontextualization with just
  one face image. More specifically, we employ a face encoder with an identity
  prior to encode the input face, and then land the face representation into a
  space with an editable prior, which is constructed from celeb names. By
  incorporating identity prior and editability prior, the learned identity can be
  injected anywhere with various contexts. In addition, we design a masked
  two-phase diffusion loss to boost the pixel-level perception of the input face
  and maintain the diversity of generation. Extensive experiments demonstrate our
  method outperforms previous customization methods. In addition, the learned
  identity can be flexibly combined with the off-the-shelf modules such as
  ControlNet. Notably, to the best knowledge, we are the first to directly inject
  the identity learned from a single image into video/3D generation without
  finetuning. We believe that the proposed StableIdentity is an important step to
  unify image, video, and 3D customized generation models.
paper_ai_summary: StableIdentity injects learned identity from a single face image into various contexts, including video and 3D, ensuring identity consistency and flexible edits.
paper_ai_keywords: face encoder identity prior editable prior masked two-phase diffusion loss ControlNet identity injection video generation 3D generation identity consistency