Multi-LoRA Composition for Image Generation
Low-Rank Adaptation (LoRA) is extensively utilized in text-to-image models
for the accurate rendition of specific elements like distinct characters or
unique styles in generated images. Nonetheless, existing methods face
challenges in effectively composing multiple LoRAs, especially as the number of
LoRAs to be integrated grows, thus hindering the creation of complex imagery.
In this paper, we study multi-LoRA composition through a decoding-centric
perspective. We present two training-free methods: LoRA Switch, which
alternates between different LoRAs at each denoising step, and LoRA Composite,
which simultaneously incorporates all LoRAs to guide more cohesive image
synthesis. To evaluate the proposed approaches, we establish ComposLoRA, a new
comprehensive testbed as part of this research. It features a diverse range of
LoRA categories with 480 composition sets. Utilizing an evaluation framework
based on GPT-4V, our findings demonstrate a clear improvement in performance
with our methods over the prevalent baseline, particularly evident when
increasing the number of LoRAs in a composition.
65dd64e7ce348fc5b1214f3c
Ming Zhong
claimed_verified
2024-02-27T14:45:58.128Z
61d53df2062444ea769d3b79
/avatars/fa771202368b6b2626a8fdf1c4369239.svg
Ming Zhong
user
MingZhong
65dd64e7ce348fc5b1214f3d
Yelong Shen
admin_assigned
2024-02-27T09:34:54.510Z
6454c337a13edf669cd5d8ea
/avatars/a383a0dda7c2ef6a0d6c3c64651f42ff.svg
Yelong Shen
user
uuu6
65dd64e7ce348fc5b1214f3e
Shuohang Wang
admin_assigned
2024-02-27T09:35:03.503Z
6463b2247572c66a8e625a57
/avatars/7722fb5649d42d966ce1e478946d5f8f.svg
Wang
user
Shuohang
65dd64e7ce348fc5b1214f3f
Yadong Lu
admin_assigned
2024-02-27T09:35:15.812Z
64f570ff96185818de7a1ccb
/avatars/5d4e0a3d21fabccc7c02c441f5786a88.svg
Yadong Lu
user
Adong17
65dd64e7ce348fc5b1214f40
Yizhu Jiao
65dd64e7ce348fc5b1214f41
Siru Ouyang
admin_assigned
2024-02-27T09:35:46.215Z
64bb4eeb5ee4641b32d7db27
/avatars/fe32fe5e5f2914b46f1d05d1a67d4314.svg
Siru Ouyang
user
siruo2
65dd64e7ce348fc5b1214f42
Donghan Yu
admin_assigned
2024-02-27T09:36:45.862Z
65b01b8a29ae836e9ed5af24
/avatars/a8b78a4b54d3f10858c5925521357001.svg
Donghan Yu
user
donghanyu
65dd64e7ce348fc5b1214f43
Jiawei Han
65dd64e7ce348fc5b1214f44
Weizhu Chen
admin_assigned
2024-02-27T09:37:08.279Z
64da876370446182be5b608d
/avatars/e412fdc71404ecdf638e416846e3ebfb.svg
Weizhu Chen
user
chenweizhu
Multi-LoRA Composition for Image Generation
Low-Rank Adaptation (LoRA) is extensively utilized in text-to-image models
for the accurate rendition of specific elements like distinct characters or
unique styles in generated images. Nonetheless, existing methods face
challenges in effectively composing multiple LoRAs, especially as the number of
LoRAs to be integrated grows, thus hindering the creation of complex imagery.
In this paper, we study multi-LoRA composition through a decoding-centric
perspective. We present two training-free methods: LoRA Switch, which
alternates between different LoRAs at each denoising step, and LoRA Composite,
which simultaneously incorporates all LoRAs to guide more cohesive image
synthesis. To evaluate the proposed approaches, we establish ComposLoRA, a new
comprehensive testbed as part of this research. It features a diverse range of
LoRA categories with 480 composition sets. Utilizing an evaluation framework
based on GPT-4V, our findings demonstrate a clear improvement in performance
with our methods over the prevalent baseline, particularly evident when
increasing the number of LoRAs in a composition.
Two training-free methods, LoRA Switch and LoRA Composite, enhance multi-LoRA composition in text-to-image models, leading to improved image synthesis performance.
Low-Rank Adaptation LoRA multi-LoRA composition denoising step image synthesis ComposLoRA GPT-4V