title: AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image
    Collections
summary: Previous animatable 3D-aware GANs for human generation have primarily focused
  on either the human head or full body. However, head-only videos are relatively
  uncommon in real life, and full body generation typically does not deal with
  facial expression control and still has challenges in generating high-quality
  results. Towards applicable video avatars, we present an animatable 3D-aware
  GAN that generates portrait images with controllable facial expression, head
  pose, and shoulder movements. It is a generative model trained on unstructured
  2D image collections without using 3D or video data. For the new task, we base
  our method on the generative radiance manifold representation and equip it with
  learnable facial and head-shoulder deformations. A dual-camera rendering and
  adversarial learning scheme is proposed to improve the quality of the generated
  faces, which is critical for portrait images. A pose deformation processing
  network is developed to generate plausible deformations for challenging regions
  such as long hair. Experiments show that our method, trained on unstructured 2D
  images, can generate diverse and high-quality 3D portraits with desired control
  over different properties.
paper_authors._id: 64f83583baa3b4ec4e4711a7
paper_authors.name: Yue Wu
paper_authors._id: 64f83583baa3b4ec4e4711a8
paper_authors.name: Sicheng Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T09:10:31.894Z
paper_authors._id: 641d402c043963b1c0a578b8
paper_authors.avatarUrl: /avatars/d1a7297ca7d4cdbbe484a54d7d223551.svg
paper_authors.fullname: Sicheng Xu
paper_authors.type: user
paper_authors.user: sicxu
paper_authors._id: 64f83583baa3b4ec4e4711a9
paper_authors.name: Jianfeng Xiang
paper_authors._id: 64f83583baa3b4ec4e4711aa
paper_authors.name: Fangyun Wei
paper_authors._id: 64f83583baa3b4ec4e4711ab
paper_authors.name: Qifeng Chen
paper_authors._id: 64f83583baa3b4ec4e4711ac
paper_authors.name: Jiaolong Yang
paper_authors._id: 64f83583baa3b4ec4e4711ad
paper_authors.name: Xin Tong
paper_title: AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image
    Collections
paper_summary: Previous animatable 3D-aware GANs for human generation have primarily focused
  on either the human head or full body. However, head-only videos are relatively
  uncommon in real life, and full body generation typically does not deal with
  facial expression control and still has challenges in generating high-quality
  results. Towards applicable video avatars, we present an animatable 3D-aware
  GAN that generates portrait images with controllable facial expression, head
  pose, and shoulder movements. It is a generative model trained on unstructured
  2D image collections without using 3D or video data. For the new task, we base
  our method on the generative radiance manifold representation and equip it with
  learnable facial and head-shoulder deformations. A dual-camera rendering and
  adversarial learning scheme is proposed to improve the quality of the generated
  faces, which is critical for portrait images. A pose deformation processing
  network is developed to generate plausible deformations for challenging regions
  such as long hair. Experiments show that our method, trained on unstructured 2D
  images, can generate diverse and high-quality 3D portraits with desired control
  over different properties.
paper_ai_keywords: generative radiance manifold representation learnable facial and head-shoulder deformations dual-camera rendering adversarial learning pose deformation processing network 3D-aware GAN 3D portraits