TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented
  Reality via 3D Gaussian Splatting
Realistic 3D full-body talking avatars hold great potential in AR, with
applications ranging from e-commerce live streaming to holographic
communication. Despite advances in 3D Gaussian Splatting (3DGS) for lifelike
avatar creation, existing methods struggle with fine-grained control of facial
expressions and body movements in full-body talking tasks. Additionally, they
often lack sufficient details and cannot run in real-time on mobile devices. We
present TaoAvatar, a high-fidelity, lightweight, 3DGS-based full-body talking
avatar driven by various signals. Our approach starts by creating a
personalized clothed human parametric template that binds Gaussians to
represent appearances. We then pre-train a StyleUnet-based network to handle
complex pose-dependent non-rigid deformation, which can capture high-frequency
appearance details but is too resource-intensive for mobile devices. To
overcome this, we "bake" the non-rigid deformations into a lightweight
MLP-based network using a distillation technique and develop blend shapes to
compensate for details. Extensive experiments show that TaoAvatar achieves
state-of-the-art rendering quality while running in real-time across various
devices, maintaining 90 FPS on high-definition stereo devices such as the Apple
Vision Pro.
67e132d77d88e5241c019390
Jianchuan Chen
admin_assigned
2025-03-24T12:53:32.888Z
6794e5faaae193052d6fc1b8
/avatars/357560910abdbd52392a3a2fe0202217.svg
Jianchuan Chen
user
Janaldo
67e132d77d88e5241c019391
Jingchuan Hu
67e132d77d88e5241c019392
Gaige Wang
admin_assigned
2025-03-24T12:53:54.415Z
64647d4f07e49885b4c7892d
/avatars/806b7a842d260990731f954af73896a3.svg
Wang Gaige
user
WangGaige
67e132d77d88e5241c019393
Zhonghua Jiang
67e132d77d88e5241c019394
Tiansong Zhou
admin_assigned
2025-03-24T12:54:12.524Z
649d55b91e9de7b743897ba6
/avatars/c8d3be7df5e2f7a6ffda1adcae1e7aac.svg
Zhou
user
Tiansong
67e132d77d88e5241c019395
Zhiwen Chen
admin_assigned
2025-03-24T12:54:25.569Z
6729d2b802305fd2366b30a0
/avatars/e5be25eecf5694e3b63a20c89a20653b.svg
Zhiwen Chen
user
chenzhiwen
67e132d77d88e5241c019396
Chengfei Lv
TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented
  Reality via 3D Gaussian Splatting
Realistic 3D full-body talking avatars hold great potential in AR, with
applications ranging from e-commerce live streaming to holographic
communication. Despite advances in 3D Gaussian Splatting (3DGS) for lifelike
avatar creation, existing methods struggle with fine-grained control of facial
expressions and body movements in full-body talking tasks. Additionally, they
often lack sufficient details and cannot run in real-time on mobile devices. We
present TaoAvatar, a high-fidelity, lightweight, 3DGS-based full-body talking
avatar driven by various signals. Our approach starts by creating a
personalized clothed human parametric template that binds Gaussians to
represent appearances. We then pre-train a StyleUnet-based network to handle
complex pose-dependent non-rigid deformation, which can capture high-frequency
appearance details but is too resource-intensive for mobile devices. To
overcome this, we "bake" the non-rigid deformations into a lightweight
MLP-based network using a distillation technique and develop blend shapes to
compensate for details. Extensive experiments show that TaoAvatar achieves
state-of-the-art rendering quality while running in real-time across various
devices, maintaining 90 FPS on high-definition stereo devices such as the Apple
Vision Pro.
TaoAvatar is a high-fidelity, lightweight 3DGS-based avatar that achieves real-time performance across devices while maintaining high rendering quality, leveraging distillation techniques for non-rigid deformation.
3D Gaussian Splatting 3DGS StyleUnet non-rigid deformation ML MLP distillation blend shapes real-time performance high-definition stereo devices Apple Vision Pro