Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models
We present a method to create interpretable concept sliders that enable
precise control over attributes in image generations from diffusion models. Our
approach identifies a low-rank parameter direction corresponding to one concept
while minimizing interference with other attributes. A slider is created using
a small set of prompts or sample images; thus slider directions can be created
for either textual or visual concepts. Concept Sliders are plug-and-play: they
can be composed efficiently and continuously modulated, enabling precise
control over image generation. In quantitative experiments comparing to
previous editing techniques, our sliders exhibit stronger targeted edits with
lower interference. We showcase sliders for weather, age, styles, and
expressions, as well as slider compositions. We show how sliders can transfer
latents from StyleGAN for intuitive editing of visual concepts for which
textual description is difficult. We also find that our method can help address
persistent quality issues in Stable Diffusion XL including repair of object
deformations and fixing distorted hands. Our code, data, and trained sliders
are available at https://sliders.baulab.info/
655d7ef3d00cca1e3abf4abe
Rohit Gandikota
admin_assigned
2023-11-22T12:46:26.127Z
636daf1b56c0762cfda074b5
/avatars/f44be5eb110acfa2efbd09de6b416239.svg
Rohit Gandikota
user
RohitGandikota
655d7ef3d00cca1e3abf4abf
Joanna Materzynska
admin_assigned
2023-11-22T12:46:41.855Z
60ef074b89938e400b470da8
/avatars/c914070df46fb2dce9067d0eafc88238.svg
Joanna Materzynska
user
jomat
655d7ef3d00cca1e3abf4ac0
Tingrui Zhou
claimed_verified
2023-11-29T14:55:08.555Z
6566f1b177d8a948ac929ff3
https://cdn-avatars.huggingface.co/v1/production/uploads/6566f1b177d8a948ac929ff3/ISGItmILjR1SztmrF_8c-.jpeg
Tingrui Zhou
user
teshu0
655d7ef3d00cca1e3abf4ac1
Antonio Torralba
655d7ef3d00cca1e3abf4ac2
David Bau
admin_assigned
2023-11-22T12:47:33.229Z
6214d6c01e35c843d42d1f77
/avatars/ac208cd180b4f3ed1ec367e581facfcf.svg
David Bau
user
davidbau
Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models
We present a method to create interpretable concept sliders that enable
precise control over attributes in image generations from diffusion models. Our
approach identifies a low-rank parameter direction corresponding to one concept
while minimizing interference with other attributes. A slider is created using
a small set of prompts or sample images; thus slider directions can be created
for either textual or visual concepts. Concept Sliders are plug-and-play: they
can be composed efficiently and continuously modulated, enabling precise
control over image generation. In quantitative experiments comparing to
previous editing techniques, our sliders exhibit stronger targeted edits with
lower interference. We showcase sliders for weather, age, styles, and
expressions, as well as slider compositions. We show how sliders can transfer
latents from StyleGAN for intuitive editing of visual concepts for which
textual description is difficult. We also find that our method can help address
persistent quality issues in Stable Diffusion XL including repair of object
deformations and fixing distorted hands. Our code, data, and trained sliders
are available at https://sliders.baulab.info/
Concept sliders enable precise control over image generation attributes using diffusion models by identifying low-rank parameter directions with minimal interference.
diffusion models concept sliders low-rank parameter direction interference textual concepts visual concepts continuous modulated editing StyleGAN Stable Diffusion XL repair of object deformations fixing distorted hands