PALP: Prompt Aligned Personalization of Text-to-Image Models
Content creators often aim to create personalized images using personal
subjects that go beyond the capabilities of conventional text-to-image models.
Additionally, they may want the resulting image to encompass a specific
location, style, ambiance, and more. Existing personalization methods may
compromise personalization ability or the alignment to complex textual prompts.
This trade-off can impede the fulfillment of user prompts and subject fidelity.
We propose a new approach focusing on personalization methods for a
single prompt to address this issue. We term our approach prompt-aligned
personalization. While this may seem restrictive, our method excels in
improving text alignment, enabling the creation of images with complex and
intricate prompts, which may pose a challenge for current techniques. In
particular, our method keeps the personalized model aligned with a target
prompt using an additional score distillation sampling term. We demonstrate the
versatility of our method in multi- and single-shot settings and further show
that it can compose multiple subjects or use inspiration from reference images,
such as artworks. We compare our approach quantitatively and qualitatively with
existing baselines and state-of-the-art techniques.
65a0bd799185dcca3061a974
Moab Arar
admin_assigned
2024-01-12T09:40:45.421Z
60defb17bb3e1aaa18b005bb
/avatars/086264e93d6f7ba2eab5cc1b218df4a5.svg
Moab Arar
user
moabarar
65a0bd799185dcca3061a975
Andrey Voynov
admin_assigned
2024-01-12T09:40:51.688Z
630551fd80bc5e03dad21ea6
/avatars/849e41404df698cc89c68939de45ec9a.svg
Andrey Voynov
user
avoin
65a0bd799185dcca3061a976
Amir Hertz
admin_assigned
2024-01-12T09:40:57.655Z
63071a86fd156a1d624a4c46
/avatars/bfc2a2169c080304124eb7f9ab04306d.svg
Amir Hertz
user
amirhertz
65a0bd799185dcca3061a977
Omri Avrahami
admin_assigned
2024-01-12T09:41:04.417Z
62f6ab4fc3372328414c8689
/avatars/4f728a5b70c9fe4a64e80e2b643ca620.svg
Omri Avrahami
user
omriav
65a0bd799185dcca3061a978
Shlomi Fruchter
admin_assigned
2024-01-12T10:01:53.899Z
6335ae887fcbeb615b0f9924
/avatars/60fd9305ccc56f17d6007f781c54fe2b.svg
Shlomi Fruchter
user
shlomif
65a0bd799185dcca3061a979
Yael Pritch
65a0bd799185dcca3061a97a
Daniel Cohen-Or
admin_assigned
2024-01-12T10:02:04.337Z
628507161949ebcae8e24ec3
/avatars/008ecb3daa4c8187b5f339f1176b3c39.svg
Daniel Cohen-Or
user
cohenor
65a0bd799185dcca3061a97b
Ariel Shamir
PALP: Prompt Aligned Personalization of Text-to-Image Models
Content creators often aim to create personalized images using personal
subjects that go beyond the capabilities of conventional text-to-image models.
Additionally, they may want the resulting image to encompass a specific
location, style, ambiance, and more. Existing personalization methods may
compromise personalization ability or the alignment to complex textual prompts.
This trade-off can impede the fulfillment of user prompts and subject fidelity.
We propose a new approach focusing on personalization methods for a
single prompt to address this issue. We term our approach prompt-aligned
personalization. While this may seem restrictive, our method excels in
improving text alignment, enabling the creation of images with complex and
intricate prompts, which may pose a challenge for current techniques. In
particular, our method keeps the personalized model aligned with a target
prompt using an additional score distillation sampling term. We demonstrate the
versatility of our method in multi- and single-shot settings and further show
that it can compose multiple subjects or use inspiration from reference images,
such as artworks. We compare our approach quantitatively and qualitatively with
existing baselines and state-of-the-art techniques.
Prompt-aligned personalization improves text alignment and enhances personalized image creation with complex prompts by using score distillation sampling.
prompt-aligned personalization score distillation sampling