HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image
  Models
Personalization has emerged as a prominent aspect within the field of
generative AI, enabling the synthesis of individuals in diverse contexts and
styles, while retaining high-fidelity to their identities. However, the process
of personalization presents inherent challenges in terms of time and memory
requirements. Fine-tuning each personalized model needs considerable GPU time
investment, and storing a personalized model per subject can be demanding in
terms of storage capacity. To overcome these challenges, we propose
HyperDreamBooth-a hypernetwork capable of efficiently generating a small set of
personalized weights from a single image of a person. By composing these
weights into the diffusion model, coupled with fast finetuning, HyperDreamBooth
can generate a person's face in various contexts and styles, with high subject
details while also preserving the model's crucial knowledge of diverse styles
and semantic modifications. Our method achieves personalization on faces in
roughly 20 seconds, 25x faster than DreamBooth and 125x faster than Textual
Inversion, using as few as one reference image, with the same quality and style
diversity as DreamBooth. Also our method yields a model that is 10000x smaller
than a normal DreamBooth model. Project page: https://hyperdreambooth.github.io
64b0b332c434dbcdcd81ee70
Nataniel Ruiz
claimed_verified
2023-07-31T15:13:18.673Z
6318c6e6cb116eab31d70a16
https://cdn-avatars.huggingface.co/v1/production/uploads/6318c6e6cb116eab31d70a16/yAw5XtvqaPUkPj2B6JOzz.jpeg
Nataniel
user
flavoredquark
64b0b332c434dbcdcd81ee71
Yuanzhen Li
admin_assigned
2023-07-17T07:00:40.798Z
6334ee5831a2be3938c85639
/avatars/9821af26a585f54b595ec39f061f2250.svg
Yuanzhen Li
user
yzli
64b0b332c434dbcdcd81ee72
Varun Jampani
admin_assigned
2023-07-17T07:00:57.938Z
630e7cfe3fc17ffc50f6e835
/avatars/0b742ff094a09f9374fafcd97ab9e002.svg
Varun Jampani
user
varunjampani
64b0b332c434dbcdcd81ee73
Wei Wei
64b0b332c434dbcdcd81ee74
Tingbo Hou
64b0b332c434dbcdcd81ee75
Yael Pritch
64b0b332c434dbcdcd81ee76
Neal Wadhwa
admin_assigned
2023-07-17T10:05:45.243Z
63041bb6eedc089484c82ed3
/avatars/fca4b823f748cde5f5cff1212213523d.svg
Neal Wadhwa
user
NealWadhwa
64b0b332c434dbcdcd81ee77
Michael Rubinstein
64b0b332c434dbcdcd81ee78
Kfir Aberman
HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image
  Models
Personalization has emerged as a prominent aspect within the field of
generative AI, enabling the synthesis of individuals in diverse contexts and
styles, while retaining high-fidelity to their identities. However, the process
of personalization presents inherent challenges in terms of time and memory
requirements. Fine-tuning each personalized model needs considerable GPU time
investment, and storing a personalized model per subject can be demanding in
terms of storage capacity. To overcome these challenges, we propose
HyperDreamBooth-a hypernetwork capable of efficiently generating a small set of
personalized weights from a single image of a person. By composing these
weights into the diffusion model, coupled with fast finetuning, HyperDreamBooth
can generate a person's face in various contexts and styles, with high subject
details while also preserving the model's crucial knowledge of diverse styles
and semantic modifications. Our method achieves personalization on faces in
roughly 20 seconds, 25x faster than DreamBooth and 125x faster than Textual
Inversion, using as few as one reference image, with the same quality and style
diversity as DreamBooth. Also our method yields a model that is 10000x smaller
than a normal DreamBooth model. Project page: https://hyperdreambooth.github.io
HyperDreamBooth leverages hypernetworks for efficient personalized face generation, significantly speeding up the process and reducing model size compared to existing methods.
hypernetwork diffusion model DreamBooth Textual Inversion