title: MagiCapture: High-Resolution Multi-Concept Portrait Customization
summary: Large-scale text-to-image models including Stable Diffusion are capable of
  generating high-fidelity photorealistic portrait images. There is an active
  research area dedicated to personalizing these models, aiming to synthesize
  specific subjects or styles using provided sets of reference images. However,
  despite the plausible results from these personalization methods, they tend to
  produce images that often fall short of realism and are not yet on a
  commercially viable level. This is particularly noticeable in portrait image
  generation, where any unnatural artifact in human faces is easily discernible
  due to our inherent human bias. To address this, we introduce MagiCapture, a
  personalization method for integrating subject and style concepts to generate
  high-resolution portrait images using just a few subject and style references.
  For instance, given a handful of random selfies, our fine-tuned model can
  generate high-quality portrait images in specific styles, such as passport or
  profile photos. The main challenge with this task is the absence of ground
  truth for the composed concepts, leading to a reduction in the quality of the
  final output and an identity shift of the source subject. To address these
  issues, we present a novel Attention Refocusing loss coupled with auxiliary
  priors, both of which facilitate robust learning within this weakly supervised
  learning setting. Our pipeline also includes additional post-processing steps
  to ensure the creation of highly realistic outputs. MagiCapture outperforms
  other baselines in both quantitative and qualitative evaluations and can also
  be generalized to other non-human objects.
paper_authors._id: 650277537b875426c912dd75
paper_authors.name: Junha Hyung
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-14T07:31:00.959Z
paper_authors._id: 6331ed964db0a767bbee4706
paper_authors.avatarUrl: /avatars/2ef3fa6083ca5208f4b87c91345267d9.svg
paper_authors.fullname: Junha Hyung
paper_authors.type: user
paper_authors.user: JunhaH
paper_authors._id: 650277537b875426c912dd76
paper_authors.name: Jaeyo Shin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-14T07:32:40.704Z
paper_authors._id: 64b9f86b760936217a3936d3
paper_authors.avatarUrl: /avatars/c270977c65f6ee9c14dbbabf15568a17.svg
paper_authors.fullname: Jaeyo Shin
paper_authors.type: user
paper_authors.user: j-mayo
paper_authors._id: 650277537b875426c912dd77
paper_authors.name: Jaegul Choo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-14T07:32:48.038Z
paper_authors._id: 64be3127805e5b64572da65c
paper_authors.avatarUrl: /avatars/edd3e94ba9e375827cc75b164602bcac.svg
paper_authors.fullname: Jaegul Choo
paper_authors.type: user
paper_authors.user: joyfull78
paper_title: MagiCapture: High-Resolution Multi-Concept Portrait Customization
paper_summary: Large-scale text-to-image models including Stable Diffusion are capable of
  generating high-fidelity photorealistic portrait images. There is an active
  research area dedicated to personalizing these models, aiming to synthesize
  specific subjects or styles using provided sets of reference images. However,
  despite the plausible results from these personalization methods, they tend to
  produce images that often fall short of realism and are not yet on a
  commercially viable level. This is particularly noticeable in portrait image
  generation, where any unnatural artifact in human faces is easily discernible
  due to our inherent human bias. To address this, we introduce MagiCapture, a
  personalization method for integrating subject and style concepts to generate
  high-resolution portrait images using just a few subject and style references.
  For instance, given a handful of random selfies, our fine-tuned model can
  generate high-quality portrait images in specific styles, such as passport or
  profile photos. The main challenge with this task is the absence of ground
  truth for the composed concepts, leading to a reduction in the quality of the
  final output and an identity shift of the source subject. To address these
  issues, we present a novel Attention Refocusing loss coupled with auxiliary
  priors, both of which facilitate robust learning within this weakly supervised
  learning setting. Our pipeline also includes additional post-processing steps
  to ensure the creation of highly realistic outputs. MagiCapture outperforms
  other baselines in both quantitative and qualitative evaluations and can also
  be generalized to other non-human objects.
paper_ai_summary: MagiCapture personalizes text-to-image models by integrating subject and style concepts using limited references and an Attention Refocusing loss, enhancing realism and quality in portrait image generation.
paper_ai_keywords: MagiCapture Attention Refocusing loss auxiliary priors weakly supervised learning text-to-image models portrait image generation high-resolution images subject and style references identity shift post-processing