title: LoRACLR: Contrastive Adaptation for Customization of Diffusion Models
summary: Recent advances in text-to-image customization have enabled high-fidelity,
  context-rich generation of personalized images, allowing specific concepts to
  appear in a variety of scenarios. However, current methods struggle with
  combining multiple personalized models, often leading to attribute entanglement
  or requiring separate training to preserve concept distinctiveness. We present
  LoRACLR, a novel approach for multi-concept image generation that merges
  multiple LoRA models, each fine-tuned for a distinct concept, into a single,
  unified model without additional individual fine-tuning. LoRACLR uses a
  contrastive objective to align and merge the weight spaces of these models,
  ensuring compatibility while minimizing interference. By enforcing distinct yet
  cohesive representations for each concept, LoRACLR enables efficient, scalable
  model composition for high-quality, multi-concept image synthesis. Our results
  highlight the effectiveness of LoRACLR in accurately merging multiple concepts,
  advancing the capabilities of personalized image generation.
paper_authors._id: 675bd7f6a6ba99158dd02ad4
paper_authors.name: Enis Simsar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-13T08:55:57.355Z
paper_authors._id: 63412f2add8853dc7e306a4f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nqbteeQhS2dPA5LVy7dTD.png
paper_authors.fullname: Enis Simsar
paper_authors.type: user
paper_authors.user: enisimsar
paper_authors._id: 675bd7f6a6ba99158dd02ad5
paper_authors.name: Thomas Hofmann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:18:14.211Z
paper_authors._id: 6630831ce888d89069e6276a
paper_authors.avatarUrl: /avatars/b40d00ac8978405dd2ae66166ac969ba.svg
paper_authors.fullname: Thomas Hofmann
paper_authors.type: user
paper_authors.user: thofmann
paper_authors._id: 675bd7f6a6ba99158dd02ad6
paper_authors.name: Federico Tombari
paper_authors._id: 675bd7f6a6ba99158dd02ad7
paper_authors.name: Pinar Yanardag
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:54:55.506Z
paper_authors._id: 683b9c7d4568ffd33e7334e1
paper_authors.avatarUrl: /avatars/046bd5a78a24f3bc4e1f6cb6076970e0.svg
paper_authors.fullname: Pinar Yanardag
paper_authors.type: user
paper_authors.user: Pinguar
paper_title: LoRACLR: Contrastive Adaptation for Customization of Diffusion Models
paper_summary: Recent advances in text-to-image customization have enabled high-fidelity,
  context-rich generation of personalized images, allowing specific concepts to
  appear in a variety of scenarios. However, current methods struggle with
  combining multiple personalized models, often leading to attribute entanglement
  or requiring separate training to preserve concept distinctiveness. We present
  LoRACLR, a novel approach for multi-concept image generation that merges
  multiple LoRA models, each fine-tuned for a distinct concept, into a single,
  unified model without additional individual fine-tuning. LoRACLR uses a
  contrastive objective to align and merge the weight spaces of these models,
  ensuring compatibility while minimizing interference. By enforcing distinct yet
  cohesive representations for each concept, LoRACLR enables efficient, scalable
  model composition for high-quality, multi-concept image synthesis. Our results
  highlight the effectiveness of LoRACLR in accurately merging multiple concepts,
  advancing the capabilities of personalized image generation.
paper_ai_summary: LoRACLR merges multiple LoRA models efficiently to generate high-quality images with distinct and cohesive multi-concept representations.
paper_ai_keywords: text-to-image customization high-fidelity context-rich personalized images LoRA models contrastive objective weight spaces model composition multi-concept image synthesis