FABRIC: Personalizing Diffusion Models with Iterative Feedback
In an era where visual content generation is increasingly driven by machine
learning, the integration of human feedback into generative models presents
significant opportunities for enhancing user experience and output quality.
This study explores strategies for incorporating iterative human feedback into
the generative process of diffusion-based text-to-image models. We propose
FABRIC, a training-free approach applicable to a wide range of popular
diffusion models, which exploits the self-attention layer present in the most
widely used architectures to condition the diffusion process on a set of
feedback images. To ensure a rigorous assessment of our approach, we introduce
a comprehensive evaluation methodology, offering a robust mechanism to quantify
the performance of generative visual models that integrate human feedback. We
show that generation results improve over multiple rounds of iterative feedback
through exhaustive analysis, implicitly optimizing arbitrary user preferences.
The potential applications of these findings extend to fields such as
personalized content creation and customization.
64b8ab546a68a9a715e0c83a
Dimitri von Rütte
claimed_verified
2023-07-20T09:52:13.138Z
640501091a3babee78e41e99
https://cdn-avatars.huggingface.co/v1/production/uploads/1678365850657-640501091a3babee78e41e99.jpeg
Dimitri
user
dvruette
64b8ab546a68a9a715e0c83b
Elisabetta Fedele
claimed_verified
2023-07-20T07:16:35.271Z
62df0ae635a8bfe88fc4ac80
https://cdn-avatars.huggingface.co/v1/production/uploads/62df0ae635a8bfe88fc4ac80/yWu5Kfq_88Ni2uy3HvnLe.jpeg
elisabetta fedele
user
efedele
64b8ab546a68a9a715e0c83c
Jonathan Thomm
claimed_verified
2023-07-20T09:52:10.016Z
63dec36ee742e86dc920f2cb
/avatars/3c5fd4ab5e5286dbf948ca113f7cbfb4.svg
Jonathan Thomm
user
jthomm
64b8ab546a68a9a715e0c83d
Lukas Wolf
claimed_verified
2023-07-20T11:01:07.762Z
63ea0c6e2e476376eb08b9fe
/avatars/a92b5261bd090f142945b60bdba19714.svg
Lukas 
user
luwo
FABRIC: Personalizing Diffusion Models with Iterative Feedback
In an era where visual content generation is increasingly driven by machine
learning, the integration of human feedback into generative models presents
significant opportunities for enhancing user experience and output quality.
This study explores strategies for incorporating iterative human feedback into
the generative process of diffusion-based text-to-image models. We propose
FABRIC, a training-free approach applicable to a wide range of popular
diffusion models, which exploits the self-attention layer present in the most
widely used architectures to condition the diffusion process on a set of
feedback images. To ensure a rigorous assessment of our approach, we introduce
a comprehensive evaluation methodology, offering a robust mechanism to quantify
the performance of generative visual models that integrate human feedback. We
show that generation results improve over multiple rounds of iterative feedback
through exhaustive analysis, implicitly optimizing arbitrary user preferences.
The potential applications of these findings extend to fields such as
personalized content creation and customization.
FABRIC is a training-free method that uses feedback images and self-attention layers to enhance diffusion-based text-to-image models through iterative human feedback.
diffusion-based text-to-image models FABRIC self-attention layers iterative human feedback generative visual models