StyleStudio: Text-Driven Style Transfer with Selective Control of Style
  Elements
Text-driven style transfer aims to merge the style of a reference image with
content described by a text prompt. Recent advancements in text-to-image models
have improved the nuance of style transformations, yet significant challenges
remain, particularly with overfitting to reference styles, limiting stylistic
control, and misaligning with textual content. In this paper, we propose three
complementary strategies to address these issues. First, we introduce a
cross-modal Adaptive Instance Normalization (AdaIN) mechanism for better
integration of style and text features, enhancing alignment. Second, we develop
a Style-based Classifier-Free Guidance (SCFG) approach that enables selective
control over stylistic elements, reducing irrelevant influences. Finally, we
incorporate a teacher model during early generation stages to stabilize spatial
layouts and mitigate artifacts. Our extensive evaluations demonstrate
significant improvements in style transfer quality and alignment with textual
prompts. Furthermore, our approach can be integrated into existing style
transfer frameworks without fine-tuning.
675a91605f94292180d7856a
Mingkun Lei
claimed_verified
2024-12-13T15:23:06.486Z
65b705e9cc412b887626f459
https://cdn-avatars.huggingface.co/v1/production/uploads/65b705e9cc412b887626f459/wZENkgjoDiuvgIGnv3mKe.jpeg
Mingkun Lei
user
Leimingkun
675a91605f94292180d7856b
Xue Song
admin_assigned
2024-12-18T13:43:51.111Z
649105f787f76be807eb7fa9
/avatars/089f1a6bdd2174839227beef792cf6ed.svg
song
user
xuesong0714
675a91605f94292180d7856c
Beier Zhu
admin_assigned
2024-12-18T13:43:57.961Z
65b77f013426ed9b26be4e36
/avatars/8208c0df81ec15defa20a7ca09481575.svg
Beier Zhu
user
BeierZ
675a91605f94292180d7856d
Hao Wang
675a91605f94292180d7856e
Chi Zhang
StyleStudio: Text-Driven Style Transfer with Selective Control of Style
  Elements
Text-driven style transfer aims to merge the style of a reference image with
content described by a text prompt. Recent advancements in text-to-image models
have improved the nuance of style transformations, yet significant challenges
remain, particularly with overfitting to reference styles, limiting stylistic
control, and misaligning with textual content. In this paper, we propose three
complementary strategies to address these issues. First, we introduce a
cross-modal Adaptive Instance Normalization (AdaIN) mechanism for better
integration of style and text features, enhancing alignment. Second, we develop
a Style-based Classifier-Free Guidance (SCFG) approach that enables selective
control over stylistic elements, reducing irrelevant influences. Finally, we
incorporate a teacher model during early generation stages to stabilize spatial
layouts and mitigate artifacts. Our extensive evaluations demonstrate
significant improvements in style transfer quality and alignment with textual
prompts. Furthermore, our approach can be integrated into existing style
transfer frameworks without fine-tuning.
The paper proposes a multi-strategy approach to enhance text-driven style transfer by improving style integration, stylistic control, and artifact mitigation.
Adaptive Instance Normalization AdaIN Style-based Classifier-Free Guidance SCFG teacher model style integration stylistic control artifact mitigation text-driven style transfer