title: Inserting Anybody in Diffusion Models via Celeb Basis
summary: Exquisite demand exists for customizing the pretrained large text-to-image
  model, e.g., Stable Diffusion, to generate innovative concepts, such
  as the users themselves. However, the newly-added concept from previous
  customization methods often shows weaker combination abilities than the
  original ones even given several images during training. We thus propose a new
  personalization method that allows for the seamless integration of a unique
  individual into the pre-trained diffusion model using just one facial
  photograph and only 1024 learnable parameters under 3
  minutes. So as we can effortlessly generate stunning images of this person in
  any pose or position, interacting with anyone and doing anything imaginable
  from text prompts. To achieve this, we first analyze and build a well-defined
  celeb basis from the embedding space of the pre-trained large text encoder.
  Then, given one facial photo as the target identity, we generate its own
  embedding by optimizing the weight of this basis and locking all other
  parameters. Empowered by the proposed celeb basis, the new identity in our
  customized model showcases a better concept combination ability than previous
  personalization methods. Besides, our model can also learn several new
  identities at once and interact with each other where the previous
  customization model fails to. The code will be released.
paper_authors._id: 64797cd85fa05d12dc35b988
paper_authors.name: Ge Yuan
paper_authors._id: 64797cd85fa05d12dc35b989
paper_authors.name: Xiaodong Cun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T08:05:39.087Z
paper_authors._id: 63184c517ca1b876d99b7e0e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63184c517ca1b876d99b7e0e/b-qDExoeJuDXK0cJBZKnz.jpeg
paper_authors.fullname: Xiaodong Cun
paper_authors.type: user
paper_authors.user: vinthony
paper_authors._id: 64797cd85fa05d12dc35b98a
paper_authors.name: Yong Zhang
paper_authors._id: 64797cd85fa05d12dc35b98b
paper_authors.name: Maomao Li
paper_authors._id: 64797cd85fa05d12dc35b98c
paper_authors.name: Chenyang Qi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T08:06:56.908Z
paper_authors._id: 642157a201c62c1e41fc526a
paper_authors.avatarUrl: /avatars/73ae831a73947a04468b1e87d17da24d.svg
paper_authors.fullname: chenyangqi
paper_authors.type: user
paper_authors.user: chenyangqi
paper_authors._id: 64797cd85fa05d12dc35b98d
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T08:07:19.530Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 64797cd85fa05d12dc35b98e
paper_authors.name: Ying Shan
paper_authors._id: 64797cd85fa05d12dc35b98f
paper_authors.name: Huicheng Zheng
paper_title: Inserting Anybody in Diffusion Models via Celeb Basis
paper_summary: Exquisite demand exists for customizing the pretrained large text-to-image
  model, e.g., Stable Diffusion, to generate innovative concepts, such
  as the users themselves. However, the newly-added concept from previous
  customization methods often shows weaker combination abilities than the
  original ones even given several images during training. We thus propose a new
  personalization method that allows for the seamless integration of a unique
  individual into the pre-trained diffusion model using just one facial
  photograph and only 1024 learnable parameters under 3
  minutes. So as we can effortlessly generate stunning images of this person in
  any pose or position, interacting with anyone and doing anything imaginable
  from text prompts. To achieve this, we first analyze and build a well-defined
  celeb basis from the embedding space of the pre-trained large text encoder.
  Then, given one facial photo as the target identity, we generate its own
  embedding by optimizing the weight of this basis and locking all other
  parameters. Empowered by the proposed celeb basis, the new identity in our
  customized model showcases a better concept combination ability than previous
  personalization methods. Besides, our model can also learn several new
  identities at once and interact with each other where the previous
  customization model fails to. The code will be released.
paper_ai_summary: A novel method personalizes pre-trained diffusion models using minimal data and parameters, enabling high-quality image generation of new identities and interactions.
paper_ai_keywords: pretrained large text-to-image model Stable Diffusion diffusion model embedding space text encoder celeb basis image generation customization method concept combination ability identity interaction