title: LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style
    Conditioned Image Generation
summary: Recent advancements in image generation models have enabled personalized
  image creation with both user-defined subjects (content) and styles. Prior
  works achieved personalization by merging corresponding low-rank adaptation
  parameters (LoRAs) through optimization-based methods, which are
  computationally demanding and unsuitable for real-time use on
  resource-constrained devices like smartphones. To address this, we introduce
  LoRA.rar, a method that not only improves image quality but also achieves a
  remarkable speedup of over 4000times in the merging process. LoRA.rar
  pre-trains a hypernetwork on a diverse set of content-style LoRA pairs,
  learning an efficient merging strategy that generalizes to new, unseen
  content-style pairs, enabling fast, high-quality personalization. Moreover, we
  identify limitations in existing evaluation metrics for content-style quality
  and propose a new protocol using multimodal large language models (MLLM) for
  more accurate assessment. Our method significantly outperforms the current
  state of the art in both content and style fidelity, as validated by MLLM
  assessments and human evaluations.
paper_authors._id: 67570c548e4eeb4b6f040bef
paper_authors.name: Donald Shenaj
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:20:52.312Z
paper_authors._id: 651ed00e8ed730cc0066c2fb
paper_authors.avatarUrl: /avatars/2790728f588075cf767a0db157115d19.svg
paper_authors.fullname: Donald Shenaj
paper_authors.type: user
paper_authors.user: donaldssh
paper_authors._id: 67570c548e4eeb4b6f040bf0
paper_authors.name: Ondrej Bohdal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T09:20:49.872Z
paper_authors._id: 65b11831c9a5a7680faa8cc9
paper_authors.avatarUrl: /avatars/e14a0f7ee2883926e7f6e146365c7ccf.svg
paper_authors.fullname: Ondrej Bohdal
paper_authors.type: user
paper_authors.user: obohdal
paper_authors._id: 67570c548e4eeb4b6f040bf1
paper_authors.name: Mete Ozay
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:50:54.799Z
paper_authors._id: 65bced24c63d6a8d7f400b8b
paper_authors.avatarUrl: /avatars/91212d7f39eba91158d7a0d0c823bc45.svg
paper_authors.fullname: Mete Ozay
paper_authors.type: user
paper_authors.user: mosams
paper_authors._id: 67570c548e4eeb4b6f040bf2
paper_authors.name: Pietro Zanuttigh
paper_authors._id: 67570c548e4eeb4b6f040bf3
paper_authors.name: Umberto Michieli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:51:08.508Z
paper_authors._id: 646f1879c261dc4133830dd6
paper_authors.avatarUrl: /avatars/f4134a8fb171f93c098a43f1c7a94eb6.svg
paper_authors.fullname: Umberto Michieli
paper_authors.type: user
paper_authors.user: u-michieli
paper_title: LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style
    Conditioned Image Generation
paper_summary: Recent advancements in image generation models have enabled personalized
  image creation with both user-defined subjects (content) and styles. Prior
  works achieved personalization by merging corresponding low-rank adaptation
  parameters (LoRAs) through optimization-based methods, which are
  computationally demanding and unsuitable for real-time use on
  resource-constrained devices like smartphones. To address this, we introduce
  LoRA.rar, a method that not only improves image quality but also achieves a
  remarkable speedup of over 4000times in the merging process. LoRA.rar
  pre-trains a hypernetwork on a diverse set of content-style LoRA pairs,
  learning an efficient merging strategy that generalizes to new, unseen
  content-style pairs, enabling fast, high-quality personalization. Moreover, we
  identify limitations in existing evaluation metrics for content-style quality
  and propose a new protocol using multimodal large language models (MLLM) for
  more accurate assessment. Our method significantly outperforms the current
  state of the art in both content and style fidelity, as validated by MLLM
  assessments and human evaluations.
paper_ai_keywords: low-rank adaptation (LoRAs) hypernetwork merging strategy multimodal large language models (MLLM) content-style quality evaluation metrics content and style fidelity fast, high-quality personalization