ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity
  Preserving
Diffusion-based technologies have made significant strides, particularly in
personalized and customized facialgeneration. However, existing methods face
challenges in achieving high-fidelity and detailed identity (ID)consistency,
primarily due to insufficient fine-grained control over facial areas and the
lack of a comprehensive strategy for ID preservation by fully considering
intricate facial details and the overall face. To address these limitations, we
introduce ConsistentID, an innovative method crafted for
diverseidentity-preserving portrait generation under fine-grained multimodal
facial prompts, utilizing only a single reference image. ConsistentID comprises
two key components: a multimodal facial prompt generator that combines facial
features, corresponding facial descriptions and the overall facial context to
enhance precision in facial details, and an ID-preservation network optimized
through the facial attention localization strategy, aimed at preserving ID
consistency in facial regions. Together, these components significantly enhance
the accuracy of ID preservation by introducing fine-grained multimodal ID
information from facial regions. To facilitate training of ConsistentID, we
present a fine-grained portrait dataset, FGID, with over 500,000 facial images,
offering greater diversity and comprehensiveness than existing public facial
datasets. % such as LAION-Face, CelebA, FFHQ, and SFHQ. Experimental results
substantiate that our ConsistentID achieves exceptional precision and diversity
in personalized facial generation, surpassing existing methods in the MyStyle
dataset. Furthermore, while ConsistentID introduces more multimodal ID
information, it maintains a fast inference speed during generation.
662b20e08bccd5f54428fcab
Jiehui Huang
claimed_verified
2024-04-26T11:56:55.893Z
64d1efef4a204a4d125fd4fc
https://cdn-avatars.huggingface.co/v1/production/uploads/64d1efef4a204a4d125fd4fc/mOjMzPUkAR-SmzYwofazX.jpeg
JackAILab
user
JackAILab
662b20e08bccd5f54428fcac
Xiao Dong
662b20e08bccd5f54428fcad
Wenhui Song
662b20e08bccd5f54428fcae
Hanhui Li
662b20e08bccd5f54428fcaf
Jun Zhou
662b20e08bccd5f54428fcb0
Yuhao Cheng
admin_assigned
2024-04-26T10:22:45.382Z
64b741376c169983c9804c21
https://cdn-avatars.huggingface.co/v1/production/uploads/64b741376c169983c9804c21/C1L8svFi3InAiCcSBOY8x.jpeg
Yuhao Cheng
user
Rovio
662b20e08bccd5f54428fcb1
Shutao Liao
admin_assigned
2024-04-26T10:23:08.528Z
65e8a6cab406b73958a799ab
/avatars/0af594af009066938409fc3246c0b058.svg
Liao Shutao
user
kaideee
662b20e08bccd5f54428fcb2
Long Chen
662b20e08bccd5f54428fcb3
Yiqiang Yan
662b20e08bccd5f54428fcb4
Shengcai Liao
662b20e08bccd5f54428fcb5
Xiaodan Liang
ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity
  Preserving
Diffusion-based technologies have made significant strides, particularly in
personalized and customized facialgeneration. However, existing methods face
challenges in achieving high-fidelity and detailed identity (ID)consistency,
primarily due to insufficient fine-grained control over facial areas and the
lack of a comprehensive strategy for ID preservation by fully considering
intricate facial details and the overall face. To address these limitations, we
introduce ConsistentID, an innovative method crafted for
diverseidentity-preserving portrait generation under fine-grained multimodal
facial prompts, utilizing only a single reference image. ConsistentID comprises
two key components: a multimodal facial prompt generator that combines facial
features, corresponding facial descriptions and the overall facial context to
enhance precision in facial details, and an ID-preservation network optimized
through the facial attention localization strategy, aimed at preserving ID
consistency in facial regions. Together, these components significantly enhance
the accuracy of ID preservation by introducing fine-grained multimodal ID
information from facial regions. To facilitate training of ConsistentID, we
present a fine-grained portrait dataset, FGID, with over 500,000 facial images,
offering greater diversity and comprehensiveness than existing public facial
datasets. % such as LAION-Face, CelebA, FFHQ, and SFHQ. Experimental results
substantiate that our ConsistentID achieves exceptional precision and diversity
in personalized facial generation, surpassing existing methods in the MyStyle
dataset. Furthermore, while ConsistentID introduces more multimodal ID
information, it maintains a fast inference speed during generation.
ConsistentID, a method for identity-preserving facial generation, uses a multimodal prompt generator and an ID-preservation network to achieve high-fidelity identity consistency with fine-grained control over facial details.
diffusion-based technologies personalized facial generation ID consistency facial attention localization multimodal facial prompts ID-preservation network FGID dataset