Style-Friendly SNR Sampler for Style-Driven Generation
Recent large-scale diffusion models generate high-quality images but struggle
to learn new, personalized artistic styles, which limits the creation of unique
style templates. Fine-tuning with reference images is the most promising
approach, but it often blindly utilizes objectives and noise level
distributions used for pre-training, leading to suboptimal style alignment. We
propose the Style-friendly SNR sampler, which aggressively shifts the
signal-to-noise ratio (SNR) distribution toward higher noise levels during
fine-tuning to focus on noise levels where stylistic features emerge. This
enables models to better capture unique styles and generate images with higher
style alignment. Our method allows diffusion models to learn and share new
"style templates", enhancing personalized content creation. We demonstrate the
ability to generate styles such as personal watercolor paintings, minimal flat
cartoons, 3D renderings, multi-panel images, and memes with text, thereby
broadening the scope of style-driven generation.
6743d8895689f9445a5da3e9
Jooyoung Choi
claimed_verified
2024-11-25T08:34:18.831Z
62f9e54defd2de85492a4dde
/avatars/9925de0d02b2f2ed03cbafda44fd7de4.svg
Jooyoung Choi
user
jychoi
6743d8895689f9445a5da3ea
Chaehun Shin
claimed_verified
2024-11-25T08:34:16.800Z
631842124d24c00131a66656
/avatars/590633e163da425bf041e3988d8dd015.svg
shin
user
chaehun
6743d8895689f9445a5da3eb
Yeongtak Oh
claimed_verified
2024-11-25T08:34:15.050Z
636f6e8a31af06da86499ebc
/avatars/9430fbc05774aad8e46c0861769b3c30.svg
Yeongtak
user
Yeongtak
6743d8895689f9445a5da3ec
Heeseung Kim
admin_assigned
2024-11-25T21:17:58.967Z
63f04b085c2ceb16fc73038a
/avatars/27aff620692fcf1d877f479105dc8d76.svg
Heeseung Kim
user
gmltmd789
6743d8895689f9445a5da3ed
Sungroh Yoon
admin_assigned
2024-11-25T21:17:53.412Z
64c99967d33e6a7f82c35f0b
/avatars/70e77eb570d12e11c2807e10cd4d926d.svg
Sungroh Yoon
user
sryoon
Style-Friendly SNR Sampler for Style-Driven Generation
Recent large-scale diffusion models generate high-quality images but struggle
to learn new, personalized artistic styles, which limits the creation of unique
style templates. Fine-tuning with reference images is the most promising
approach, but it often blindly utilizes objectives and noise level
distributions used for pre-training, leading to suboptimal style alignment. We
propose the Style-friendly SNR sampler, which aggressively shifts the
signal-to-noise ratio (SNR) distribution toward higher noise levels during
fine-tuning to focus on noise levels where stylistic features emerge. This
enables models to better capture unique styles and generate images with higher
style alignment. Our method allows diffusion models to learn and share new
"style templates", enhancing personalized content creation. We demonstrate the
ability to generate styles such as personal watercolor paintings, minimal flat
cartoons, 3D renderings, multi-panel images, and memes with text, thereby
broadening the scope of style-driven generation.
The Style-friendly SNR sampler modifies the noise level distribution during fine-tuning to improve style alignment in diffusion models, enabling better capture of unique artistic styles.
diffusion models signal-to-noise ratio (SNR) fine-tuning noise level distribution stylistic features style templates personalized content creation