AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image
  Collections
Previous animatable 3D-aware GANs for human generation have primarily focused
on either the human head or full body. However, head-only videos are relatively
uncommon in real life, and full body generation typically does not deal with
facial expression control and still has challenges in generating high-quality
results. Towards applicable video avatars, we present an animatable 3D-aware
GAN that generates portrait images with controllable facial expression, head
pose, and shoulder movements. It is a generative model trained on unstructured
2D image collections without using 3D or video data. For the new task, we base
our method on the generative radiance manifold representation and equip it with
learnable facial and head-shoulder deformations. A dual-camera rendering and
adversarial learning scheme is proposed to improve the quality of the generated
faces, which is critical for portrait images. A pose deformation processing
network is developed to generate plausible deformations for challenging regions
such as long hair. Experiments show that our method, trained on unstructured 2D
images, can generate diverse and high-quality 3D portraits with desired control
over different properties.
64f83583baa3b4ec4e4711a7
Yue Wu
64f83583baa3b4ec4e4711a8
Sicheng Xu
claimed_verified
2024-12-06T09:10:31.894Z
641d402c043963b1c0a578b8
/avatars/d1a7297ca7d4cdbbe484a54d7d223551.svg
Sicheng Xu
user
sicxu
64f83583baa3b4ec4e4711a9
Jianfeng Xiang
64f83583baa3b4ec4e4711aa
Fangyun Wei
64f83583baa3b4ec4e4711ab
Qifeng Chen
64f83583baa3b4ec4e4711ac
Jiaolong Yang
64f83583baa3b4ec4e4711ad
Xin Tong
AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image
  Collections
Previous animatable 3D-aware GANs for human generation have primarily focused
on either the human head or full body. However, head-only videos are relatively
uncommon in real life, and full body generation typically does not deal with
facial expression control and still has challenges in generating high-quality
results. Towards applicable video avatars, we present an animatable 3D-aware
GAN that generates portrait images with controllable facial expression, head
pose, and shoulder movements. It is a generative model trained on unstructured
2D image collections without using 3D or video data. For the new task, we base
our method on the generative radiance manifold representation and equip it with
learnable facial and head-shoulder deformations. A dual-camera rendering and
adversarial learning scheme is proposed to improve the quality of the generated
faces, which is critical for portrait images. A pose deformation processing
network is developed to generate plausible deformations for challenging regions
such as long hair. Experiments show that our method, trained on unstructured 2D
images, can generate diverse and high-quality 3D portraits with desired control
over different properties.
The proposed 3D-aware GAN generates high-quality, controllable portrait images from unstructured 2D images, addressing challenges in facial and head pose representation.
GAN generative radiance manifold learnable deformations dual-camera rendering adversarial learning pose deformation processing network