title: ConsisLoRA: Enhancing Content and Style Consistency for LoRA-based Style
    Transfer
summary: Style transfer involves transferring the style from a reference image to the
  content of a target image. Recent advancements in LoRA-based (Low-Rank
  Adaptation) methods have shown promise in effectively capturing the style of a
  single image. However, these approaches still face significant challenges such
  as content inconsistency, style misalignment, and content leakage. In this
  paper, we comprehensively analyze the limitations of the standard diffusion
  parameterization, which learns to predict noise, in the context of style
  transfer. To address these issues, we introduce ConsisLoRA, a LoRA-based method
  that enhances both content and style consistency by optimizing the LoRA weights
  to predict the original image rather than noise. We also propose a two-step
  training strategy that decouples the learning of content and style from the
  reference image. To effectively capture both the global structure and local
  details of the content image, we introduce a stepwise loss transition strategy.
  Additionally, we present an inference guidance method that enables continuous
  control over content and style strengths during inference. Through both
  qualitative and quantitative evaluations, our method demonstrates significant
  improvements in content and style consistency while effectively reducing
  content leakage.
paper_authors._id: 67d3d770336d57afb22fbe46
paper_authors.name: Bolin Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:55:56.615Z
paper_authors._id: 66e508b7ad382452bb326cbd
paper_authors.avatarUrl: /avatars/e541e312c4fe9919778d266339d38206.svg
paper_authors.fullname: bolin
paper_authors.type: user
paper_authors.user: chenblin26
paper_authors._id: 67d3d770336d57afb22fbe47
paper_authors.name: Baoquan Zhao
paper_authors._id: 67d3d770336d57afb22fbe48
paper_authors.name: Haoran Xie
paper_authors._id: 67d3d770336d57afb22fbe49
paper_authors.name: Yi Cai
paper_authors._id: 67d3d770336d57afb22fbe4a
paper_authors.name: Qing Li
paper_authors._id: 67d3d770336d57afb22fbe4b
paper_authors.name: Xudong Mao
paper_title: ConsisLoRA: Enhancing Content and Style Consistency for LoRA-based Style
    Transfer
paper_summary: Style transfer involves transferring the style from a reference image to the
  content of a target image. Recent advancements in LoRA-based (Low-Rank
  Adaptation) methods have shown promise in effectively capturing the style of a
  single image. However, these approaches still face significant challenges such
  as content inconsistency, style misalignment, and content leakage. In this
  paper, we comprehensively analyze the limitations of the standard diffusion
  parameterization, which learns to predict noise, in the context of style
  transfer. To address these issues, we introduce ConsisLoRA, a LoRA-based method
  that enhances both content and style consistency by optimizing the LoRA weights
  to predict the original image rather than noise. We also propose a two-step
  training strategy that decouples the learning of content and style from the
  reference image. To effectively capture both the global structure and local
  details of the content image, we introduce a stepwise loss transition strategy.
  Additionally, we present an inference guidance method that enables continuous
  control over content and style strengths during inference. Through both
  qualitative and quantitative evaluations, our method demonstrates significant
  improvements in content and style consistency while effectively reducing
  content leakage.
paper_ai_summary: ConsisLoRA, a LoRA-based method, improves style transfer by predicting original images and separating content from style learning, with enhanced consistency and reduced leakage.
paper_ai_keywords: LoRA ConsisLoRA diffusion parameterization content consistency style consistency style misalignment content leakage two-step training stepwise loss transition inference guidance