title: StyleStudio: Text-Driven Style Transfer with Selective Control of Style
    Elements
summary: Text-driven style transfer aims to merge the style of a reference image with
  content described by a text prompt. Recent advancements in text-to-image models
  have improved the nuance of style transformations, yet significant challenges
  remain, particularly with overfitting to reference styles, limiting stylistic
  control, and misaligning with textual content. In this paper, we propose three
  complementary strategies to address these issues. First, we introduce a
  cross-modal Adaptive Instance Normalization (AdaIN) mechanism for better
  integration of style and text features, enhancing alignment. Second, we develop
  a Style-based Classifier-Free Guidance (SCFG) approach that enables selective
  control over stylistic elements, reducing irrelevant influences. Finally, we
  incorporate a teacher model during early generation stages to stabilize spatial
  layouts and mitigate artifacts. Our extensive evaluations demonstrate
  significant improvements in style transfer quality and alignment with textual
  prompts. Furthermore, our approach can be integrated into existing style
  transfer frameworks without fine-tuning.
paper_authors._id: 675a91605f94292180d7856a
paper_authors.name: Mingkun Lei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-13T15:23:06.486Z
paper_authors._id: 65b705e9cc412b887626f459
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65b705e9cc412b887626f459/wZENkgjoDiuvgIGnv3mKe.jpeg
paper_authors.fullname: Mingkun Lei
paper_authors.type: user
paper_authors.user: Leimingkun
paper_authors._id: 675a91605f94292180d7856b
paper_authors.name: Xue Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T13:43:51.111Z
paper_authors._id: 649105f787f76be807eb7fa9
paper_authors.avatarUrl: /avatars/089f1a6bdd2174839227beef792cf6ed.svg
paper_authors.fullname: song
paper_authors.type: user
paper_authors.user: xuesong0714
paper_authors._id: 675a91605f94292180d7856c
paper_authors.name: Beier Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T13:43:57.961Z
paper_authors._id: 65b77f013426ed9b26be4e36
paper_authors.avatarUrl: /avatars/8208c0df81ec15defa20a7ca09481575.svg
paper_authors.fullname: Beier Zhu
paper_authors.type: user
paper_authors.user: BeierZ
paper_authors._id: 675a91605f94292180d7856d
paper_authors.name: Hao Wang
paper_authors._id: 675a91605f94292180d7856e
paper_authors.name: Chi Zhang
paper_title: StyleStudio: Text-Driven Style Transfer with Selective Control of Style
    Elements
paper_summary: Text-driven style transfer aims to merge the style of a reference image with
  content described by a text prompt. Recent advancements in text-to-image models
  have improved the nuance of style transformations, yet significant challenges
  remain, particularly with overfitting to reference styles, limiting stylistic
  control, and misaligning with textual content. In this paper, we propose three
  complementary strategies to address these issues. First, we introduce a
  cross-modal Adaptive Instance Normalization (AdaIN) mechanism for better
  integration of style and text features, enhancing alignment. Second, we develop
  a Style-based Classifier-Free Guidance (SCFG) approach that enables selective
  control over stylistic elements, reducing irrelevant influences. Finally, we
  incorporate a teacher model during early generation stages to stabilize spatial
  layouts and mitigate artifacts. Our extensive evaluations demonstrate
  significant improvements in style transfer quality and alignment with textual
  prompts. Furthermore, our approach can be integrated into existing style
  transfer frameworks without fine-tuning.
paper_ai_summary: The paper proposes a multi-strategy approach to enhance text-driven style transfer by improving style integration, stylistic control, and artifact mitigation.
paper_ai_keywords: Adaptive Instance Normalization AdaIN Style-based Classifier-Free Guidance SCFG teacher model style integration stylistic control artifact mitigation text-driven style transfer