MagiCapture: High-Resolution Multi-Concept Portrait Customization
Large-scale text-to-image models including Stable Diffusion are capable of
generating high-fidelity photorealistic portrait images. There is an active
research area dedicated to personalizing these models, aiming to synthesize
specific subjects or styles using provided sets of reference images. However,
despite the plausible results from these personalization methods, they tend to
produce images that often fall short of realism and are not yet on a
commercially viable level. This is particularly noticeable in portrait image
generation, where any unnatural artifact in human faces is easily discernible
due to our inherent human bias. To address this, we introduce MagiCapture, a
personalization method for integrating subject and style concepts to generate
high-resolution portrait images using just a few subject and style references.
For instance, given a handful of random selfies, our fine-tuned model can
generate high-quality portrait images in specific styles, such as passport or
profile photos. The main challenge with this task is the absence of ground
truth for the composed concepts, leading to a reduction in the quality of the
final output and an identity shift of the source subject. To address these
issues, we present a novel Attention Refocusing loss coupled with auxiliary
priors, both of which facilitate robust learning within this weakly supervised
learning setting. Our pipeline also includes additional post-processing steps
to ensure the creation of highly realistic outputs. MagiCapture outperforms
other baselines in both quantitative and qualitative evaluations and can also
be generalized to other non-human objects.
650277537b875426c912dd75
Junha Hyung
admin_assigned
2023-09-14T07:31:00.959Z
6331ed964db0a767bbee4706
/avatars/2ef3fa6083ca5208f4b87c91345267d9.svg
Junha Hyung
user
JunhaH
650277537b875426c912dd76
Jaeyo Shin
admin_assigned
2023-09-14T07:32:40.704Z
64b9f86b760936217a3936d3
/avatars/c270977c65f6ee9c14dbbabf15568a17.svg
Jaeyo Shin
user
j-mayo
650277537b875426c912dd77
Jaegul Choo
admin_assigned
2023-09-14T07:32:48.038Z
64be3127805e5b64572da65c
/avatars/edd3e94ba9e375827cc75b164602bcac.svg
Jaegul Choo
user
joyfull78
MagiCapture: High-Resolution Multi-Concept Portrait Customization
Large-scale text-to-image models including Stable Diffusion are capable of
generating high-fidelity photorealistic portrait images. There is an active
research area dedicated to personalizing these models, aiming to synthesize
specific subjects or styles using provided sets of reference images. However,
despite the plausible results from these personalization methods, they tend to
produce images that often fall short of realism and are not yet on a
commercially viable level. This is particularly noticeable in portrait image
generation, where any unnatural artifact in human faces is easily discernible
due to our inherent human bias. To address this, we introduce MagiCapture, a
personalization method for integrating subject and style concepts to generate
high-resolution portrait images using just a few subject and style references.
For instance, given a handful of random selfies, our fine-tuned model can
generate high-quality portrait images in specific styles, such as passport or
profile photos. The main challenge with this task is the absence of ground
truth for the composed concepts, leading to a reduction in the quality of the
final output and an identity shift of the source subject. To address these
issues, we present a novel Attention Refocusing loss coupled with auxiliary
priors, both of which facilitate robust learning within this weakly supervised
learning setting. Our pipeline also includes additional post-processing steps
to ensure the creation of highly realistic outputs. MagiCapture outperforms
other baselines in both quantitative and qualitative evaluations and can also
be generalized to other non-human objects.
MagiCapture personalizes text-to-image models by integrating subject and style concepts using limited references and an Attention Refocusing loss, enhancing realism and quality in portrait image generation.
MagiCapture Attention Refocusing loss auxiliary priors weakly supervised learning text-to-image models portrait image generation high-resolution images subject and style references identity shift post-processing