CosmicMan: A Text-to-Image Foundation Model for Humans
We present CosmicMan, a text-to-image foundation model specialized for
generating high-fidelity human images. Unlike current general-purpose
foundation models that are stuck in the dilemma of inferior quality and
text-image misalignment for humans, CosmicMan enables generating
photo-realistic human images with meticulous appearance, reasonable structure,
and precise text-image alignment with detailed dense descriptions. At the heart
of CosmicMan's success are the new reflections and perspectives on data and
models: (1) We found that data quality and a scalable data production flow are
essential for the final results from trained models. Hence, we propose a new
data production paradigm, Annotate Anyone, which serves as a perpetual data
flywheel to produce high-quality data with accurate yet cost-effective
annotations over time. Based on this, we constructed a large-scale dataset,
CosmicMan-HQ 1.0, with 6 Million high-quality real-world human images in a mean
resolution of 1488x1255, and attached with precise text annotations deriving
from 115 Million attributes in diverse granularities. (2) We argue that a
text-to-image foundation model specialized for humans must be pragmatic -- easy
to integrate into down-streaming tasks while effective in producing
high-quality human images. Hence, we propose to model the relationship between
dense text descriptions and image pixels in a decomposed manner, and present
Decomposed-Attention-Refocusing (Daring) training framework. It seamlessly
decomposes the cross-attention features in existing text-to-image diffusion
model, and enforces attention refocusing without adding extra modules. Through
Daring, we show that explicitly discretizing continuous text space into several
basic groups that align with human body structure is the key to tackling the
misalignment problem in a breeze.
660c1c7db8036e5366912dc5
Shikai Li
660c1c7db8036e5366912dc6
Jianglin Fu
660c1c7db8036e5366912dc7
Kaiyuan Liu
660c1c7db8036e5366912dc8
Wentao Wang
660c1c7db8036e5366912dc9
Kwan-Yee Lin
660c1c7db8036e5366912dca
Wayne Wu
CosmicMan: A Text-to-Image Foundation Model for Humans
We present CosmicMan, a text-to-image foundation model specialized for
generating high-fidelity human images. Unlike current general-purpose
foundation models that are stuck in the dilemma of inferior quality and
text-image misalignment for humans, CosmicMan enables generating
photo-realistic human images with meticulous appearance, reasonable structure,
and precise text-image alignment with detailed dense descriptions. At the heart
of CosmicMan's success are the new reflections and perspectives on data and
models: (1) We found that data quality and a scalable data production flow are
essential for the final results from trained models. Hence, we propose a new
data production paradigm, Annotate Anyone, which serves as a perpetual data
flywheel to produce high-quality data with accurate yet cost-effective
annotations over time. Based on this, we constructed a large-scale dataset,
CosmicMan-HQ 1.0, with 6 Million high-quality real-world human images in a mean
resolution of 1488x1255, and attached with precise text annotations deriving
from 115 Million attributes in diverse granularities. (2) We argue that a
text-to-image foundation model specialized for humans must be pragmatic -- easy
to integrate into down-streaming tasks while effective in producing
high-quality human images. Hence, we propose to model the relationship between
dense text descriptions and image pixels in a decomposed manner, and present
Decomposed-Attention-Refocusing (Daring) training framework. It seamlessly
decomposes the cross-attention features in existing text-to-image diffusion
model, and enforces attention refocusing without adding extra modules. Through
Daring, we show that explicitly discretizing continuous text space into several
basic groups that align with human body structure is the key to tackling the
misalignment problem in a breeze.
CosmicMan generates high-fidelity human images using a specialized data production paradigm and Decomposed-Attention-Refocusing framework within a text-to-image diffusion model.
text-to-image foundation model photo-realistic meticulous appearance text-image alignment data production flow Annotate Anyone data flywheel CosmicMan-HQ 1.0 high-quality real-world human images text annotations dense text descriptions Decomposed-Attention-Refocusing Daring cross-attention text-to-image diffusion model misalignment problem