  Existing text-to-image synthesis methods generally are only applicable to
words in the training dataset. However, human faces are so variable to be
described with limited words. So this paper proposes the first free-style
text-to-face method namely AnyFace enabling much wider open world applications
such as metaverse, social media, cosmetics, forensics, etc. AnyFace has a novel
two-stream framework for face image synthesis and manipulation given arbitrary
descriptions of the human face. Specifically, one stream performs text-to-face
generation and the other conducts face image reconstruction. Facial text and
image features are extracted using the CLIP (Contrastive Language-Image
Pre-training) encoders. And a collaborative Cross Modal Distillation (CMD)
module is designed to align the linguistic and visual features across these two
streams. Furthermore, a Diverse Triplet Loss (DT loss) is developed to model
fine-grained features and improve facial diversity. Extensive experiments on
Multi-modal CelebA-HQ and CelebAText-HQ demonstrate significant advantages of
AnyFace over state-of-the-art methods. AnyFace can achieve high-quality,
high-resolution, and high-diversity face synthesis and manipulation results
without any constraints on the number and content of input captions.

Computer Vision and Pattern Recognition