title: UniF^2ace: Fine-grained Face Understanding and Generation
    with Unified Multimodal Models
summary: Unified multimodal models (UMMs) have emerged as a powerful paradigm in
  foundational computer vision research, demonstrating significant potential in
  both image understanding and generation. However, existing research in the face
  domain primarily focuses on coarse facial attribute understanding,
  with limited capacity to handle fine-grained facial attributes and
  without addressing generation capabilities. To overcome these limitations, we
  propose UniF^2ace, the first UMM tailored specifically for
  fine-grained face understanding and generation. In general, we train
  UniF^2ace on a self-constructed, specialized dataset utilizing two
  mutually beneficial diffusion techniques and a two-level mixture-of-experts
  architecture. Specifically, we first build a large-scale facial dataset,
  UniF^2ace-130K, which contains 130K image-text pairs with one
  million question-answering pairs that span a wide range of facial attributes.
  Second, we establish a theoretical connection between discrete diffusion score
  matching and masked generative models, optimizing both evidence lower bounds
  simultaneously, which significantly improves the model's ability to synthesize
  facial details. Finally, we introduce both token-level and sequence-level
  mixture-of-experts, enabling efficient fine-grained representation learning for
  both understanding and generation tasks. Extensive experiments on
  UniF^2ace-130K demonstrate that UniF^2ace outperforms
  existing UMMs and generative models, achieving superior performance across both
  understanding and generation tasks.
paper_authors._id: 67d0f5ace3c8042929eea946
paper_authors.name: Junzhe Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:39:26.197Z
paper_authors._id: 64c860d23a3f428da65ea499
paper_authors.avatarUrl: /avatars/f0bcc6ae7e558babe691b6bbf1059c9d.svg
paper_authors.fullname: lijunzhe
paper_authors.type: user
paper_authors.user: tulvgengenr
paper_authors._id: 67d0f5ace3c8042929eea947
paper_authors.name: Xuerui Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:38:04.042Z
paper_authors._id: 66d98c2bc9758ee602cb2e22
paper_authors.avatarUrl: /avatars/cea03f263d5ed292c4e558871112c61c.svg
paper_authors.fullname: Qiu
paper_authors.type: user
paper_authors.user: Xuerui123
paper_authors._id: 67d0f5ace3c8042929eea948
paper_authors.name: Linrui Xu
paper_authors._id: 67d0f5ace3c8042929eea949
paper_authors.name: Liya Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-12T15:38:31.340Z
paper_authors._id: 67d106a055363e24d8c4511e
paper_authors.avatarUrl: /avatars/ac3bd0d9632799e5ccb8fd96bf739e42.svg
paper_authors.fullname: Liya Guo
paper_authors.type: user
paper_authors.user: LiyaGuo66
paper_authors._id: 67d0f5ace3c8042929eea94a
paper_authors.name: Delin Qu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:39:29.349Z
paper_authors._id: 64daecec888b7e9c400f59b5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64daecec888b7e9c400f59b5/f4pfOfWk6jYJX-Nf2-qHn.png
paper_authors.fullname: Delin Qu
paper_authors.type: user
paper_authors.user: delinqu
paper_authors._id: 67d0f5ace3c8042929eea94b
paper_authors.name: Tingting Long
paper_authors._id: 67d0f5ace3c8042929eea94c
paper_authors.name: Chun Fan
paper_authors._id: 67d0f5ace3c8042929eea94d
paper_authors.name: Ming Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T14:25:54.054Z
paper_authors._id: 63842a11e58a1678ad7669c1
paper_authors.avatarUrl: /avatars/556f82d7074734a5829afaf9147d5493.svg
paper_authors.fullname: ming li
paper_authors.type: user
paper_authors.user: Frankboy
paper_title: UniF^2ace: Fine-grained Face Understanding and Generation
    with Unified Multimodal Models
paper_summary: Unified multimodal models (UMMs) have emerged as a powerful paradigm in
  foundational computer vision research, demonstrating significant potential in
  both image understanding and generation. However, existing research in the face
  domain primarily focuses on coarse facial attribute understanding,
  with limited capacity to handle fine-grained facial attributes and
  without addressing generation capabilities. To overcome these limitations, we
  propose UniF^2ace, the first UMM tailored specifically for
  fine-grained face understanding and generation. In general, we train
  UniF^2ace on a self-constructed, specialized dataset utilizing two
  mutually beneficial diffusion techniques and a two-level mixture-of-experts
  architecture. Specifically, we first build a large-scale facial dataset,
  UniF^2ace-130K, which contains 130K image-text pairs with one
  million question-answering pairs that span a wide range of facial attributes.
  Second, we establish a theoretical connection between discrete diffusion score
  matching and masked generative models, optimizing both evidence lower bounds
  simultaneously, which significantly improves the model's ability to synthesize
  facial details. Finally, we introduce both token-level and sequence-level
  mixture-of-experts, enabling efficient fine-grained representation learning for
  both understanding and generation tasks. Extensive experiments on
  UniF^2ace-130K demonstrate that UniF^2ace outperforms
  existing UMMs and generative models, achieving superior performance across both
  understanding and generation tasks.
paper_ai_summary: UniF<sup>2</sup>ace is a unified multimodal model that excels in fine-grained face understanding and generation using a two-level mixture-of-experts and diffusion techniques.
paper_ai_keywords: unified multimodal models UMMs UniF<sup>2</sup>ace fine-grained face understanding coarse facial attribute understanding UniF<sup>2</sup>ace-130K diffusion techniques discrete diffusion score matching masked generative models evidence lower bounds mixture-of-experts