title: ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
summary: Methods for finetuning generative models for concept-driven personalization
  generally achieve strong results for subject-driven or style-driven generation.
  Recently, low-rank adaptations (LoRA) have been proposed as a
  parameter-efficient way of achieving concept-driven personalization. While
  recent work explores the combination of separate LoRAs to achieve joint
  generation of learned styles and subjects, existing techniques do not reliably
  address the problem; they often compromise either subject fidelity or style
  fidelity. We propose ZipLoRA, a method to cheaply and effectively merge
  independently trained style and subject LoRAs in order to achieve generation of
  any user-provided subject in any user-provided style. Experiments on a wide
  range of subject and style combinations show that ZipLoRA can generate
  compelling results with meaningful improvements over baselines in subject and
  style fidelity while preserving the ability to recontextualize. Project page:
  https://ziplora.github.io
paper_authors._id: 655ef0c7337a5d2e37b08c23
paper_authors.name: Viraj Shah
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-23T10:55:03.035Z
paper_authors._id: 645c1b97081be4b320478149
paper_authors.avatarUrl: /avatars/d2714416c008c51feb11ea2e67cdc98e.svg
paper_authors.fullname: Viraj Shah
paper_authors.type: user
paper_authors.user: shahviraj
paper_authors._id: 655ef0c7337a5d2e37b08c24
paper_authors.name: Nataniel Ruiz
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-05T08:41:42.775Z
paper_authors._id: 6318c6e6cb116eab31d70a16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6318c6e6cb116eab31d70a16/yAw5XtvqaPUkPj2B6JOzz.jpeg
paper_authors.fullname: Nataniel
paper_authors.type: user
paper_authors.user: flavoredquark
paper_authors._id: 655ef0c7337a5d2e37b08c25
paper_authors.name: Forrester Cole
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-23T10:54:08.066Z
paper_authors._id: 635003011d81beb8e244e68d
paper_authors.avatarUrl: /avatars/32f46e6778d1c52ebdd6f2511ba48257.svg
paper_authors.fullname: Forrester Cole
paper_authors.type: user
paper_authors.user: fcole
paper_authors._id: 655ef0c7337a5d2e37b08c26
paper_authors.name: Erika Lu
paper_authors._id: 655ef0c7337a5d2e37b08c27
paper_authors.name: Svetlana Lazebnik
paper_authors._id: 655ef0c7337a5d2e37b08c28
paper_authors.name: Yuanzhen Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-23T10:54:27.631Z
paper_authors._id: 6334ee5831a2be3938c85639
paper_authors.avatarUrl: /avatars/9821af26a585f54b595ec39f061f2250.svg
paper_authors.fullname: Yuanzhen Li
paper_authors.type: user
paper_authors.user: yzli
paper_authors._id: 655ef0c7337a5d2e37b08c29
paper_authors.name: Varun Jampani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-23T10:54:35.759Z
paper_authors._id: 630e7cfe3fc17ffc50f6e835
paper_authors.avatarUrl: /avatars/0b742ff094a09f9374fafcd97ab9e002.svg
paper_authors.fullname: Varun Jampani
paper_authors.type: user
paper_authors.user: varunjampani
paper_title: ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
paper_summary: Methods for finetuning generative models for concept-driven personalization
  generally achieve strong results for subject-driven or style-driven generation.
  Recently, low-rank adaptations (LoRA) have been proposed as a
  parameter-efficient way of achieving concept-driven personalization. While
  recent work explores the combination of separate LoRAs to achieve joint
  generation of learned styles and subjects, existing techniques do not reliably
  address the problem; they often compromise either subject fidelity or style
  fidelity. We propose ZipLoRA, a method to cheaply and effectively merge
  independently trained style and subject LoRAs in order to achieve generation of
  any user-provided subject in any user-provided style. Experiments on a wide
  range of subject and style combinations show that ZipLoRA can generate
  compelling results with meaningful improvements over baselines in subject and
  style fidelity while preserving the ability to recontextualize. Project page:
  https://ziplora.github.io
paper_ai_keywords: low-rank adaptations (LoRA) parameter-efficient concept-driven personalization subject-driven style-driven independently trained ZipLoRA subject fidelity style fidelity recontextualize