ViPer: Visual Personalization of Generative Models via Individual
  Preference Learning
Different users find different images generated for the same prompt
desirable. This gives rise to personalized image generation which involves
creating images aligned with an individual's visual preference. Current
generative models are, however, unpersonalized, as they are tuned to produce
outputs that appeal to a broad audience. Using them to generate images aligned
with individual users relies on iterative manual prompt engineering by the user
which is inefficient and undesirable. We propose to personalize the image
generation process by first capturing the generic preferences of the user in a
one-time process by inviting them to comment on a small selection of images,
explaining why they like or dislike each. Based on these comments, we infer a
user's structured liked and disliked visual attributes, i.e., their visual
preference, using a large language model. These attributes are used to guide a
text-to-image model toward producing images that are tuned towards the
individual user's visual preference. Through a series of user studies and large
language model guided evaluations, we demonstrate that the proposed method
results in generations that are well aligned with individual users' visual
preferences.
66a1b4598f0d2327e0a2ce03
Sogand Salehi
claimed_verified
2024-07-25T11:07:38.414Z
63179ba4212fce5a3cd66a67
https://cdn-avatars.huggingface.co/v1/production/uploads/63179ba4212fce5a3cd66a67/qvXI90u5gWi7MP5coxkc2.png
Sogand Salehi
user
sogandstormesalehi
66a1b4598f0d2327e0a2ce04
Mahdi Shafiei
66a1b4598f0d2327e0a2ce05
Teresa Yeo
66a1b4598f0d2327e0a2ce06
Roman Bachmann
admin_assigned
2024-07-25T12:23:23.301Z
624dc3c5953e603497e3a28c
/avatars/955e0269f7e5fe3dbefeafa803a06d76.svg
Roman Bachmann
user
roman-bachmann
66a1b4598f0d2327e0a2ce07
Amir Zamir
admin_assigned
2024-07-25T12:23:28.403Z
666c7e0d29b9f769d6b2ee68
/avatars/8261e3a8e6fdd2012a2e3f91279df110.svg
Amir Zamir
user
amir32002
ViPer: Visual Personalization of Generative Models via Individual
  Preference Learning
Different users find different images generated for the same prompt
desirable. This gives rise to personalized image generation which involves
creating images aligned with an individual's visual preference. Current
generative models are, however, unpersonalized, as they are tuned to produce
outputs that appeal to a broad audience. Using them to generate images aligned
with individual users relies on iterative manual prompt engineering by the user
which is inefficient and undesirable. We propose to personalize the image
generation process by first capturing the generic preferences of the user in a
one-time process by inviting them to comment on a small selection of images,
explaining why they like or dislike each. Based on these comments, we infer a
user's structured liked and disliked visual attributes, i.e., their visual
preference, using a large language model. These attributes are used to guide a
text-to-image model toward producing images that are tuned towards the
individual user's visual preference. Through a series of user studies and large
language model guided evaluations, we demonstrate that the proposed method
results in generations that are well aligned with individual users' visual
preferences.
A method to personalize image generation using user feedback and a large language model to guide a text-to-image model.
large language model text-to-image model user feedback structured visual attributes visual preference