title: TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented
    Reality via 3D Gaussian Splatting
summary: Realistic 3D full-body talking avatars hold great potential in AR, with
  applications ranging from e-commerce live streaming to holographic
  communication. Despite advances in 3D Gaussian Splatting (3DGS) for lifelike
  avatar creation, existing methods struggle with fine-grained control of facial
  expressions and body movements in full-body talking tasks. Additionally, they
  often lack sufficient details and cannot run in real-time on mobile devices. We
  present TaoAvatar, a high-fidelity, lightweight, 3DGS-based full-body talking
  avatar driven by various signals. Our approach starts by creating a
  personalized clothed human parametric template that binds Gaussians to
  represent appearances. We then pre-train a StyleUnet-based network to handle
  complex pose-dependent non-rigid deformation, which can capture high-frequency
  appearance details but is too resource-intensive for mobile devices. To
  overcome this, we "bake" the non-rigid deformations into a lightweight
  MLP-based network using a distillation technique and develop blend shapes to
  compensate for details. Extensive experiments show that TaoAvatar achieves
  state-of-the-art rendering quality while running in real-time across various
  devices, maintaining 90 FPS on high-definition stereo devices such as the Apple
  Vision Pro.
paper_authors._id: 67e132d77d88e5241c019390
paper_authors.name: Jianchuan Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-24T12:53:32.888Z
paper_authors._id: 6794e5faaae193052d6fc1b8
paper_authors.avatarUrl: /avatars/357560910abdbd52392a3a2fe0202217.svg
paper_authors.fullname: Jianchuan Chen
paper_authors.type: user
paper_authors.user: Janaldo
paper_authors._id: 67e132d77d88e5241c019391
paper_authors.name: Jingchuan Hu
paper_authors._id: 67e132d77d88e5241c019392
paper_authors.name: Gaige Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-24T12:53:54.415Z
paper_authors._id: 64647d4f07e49885b4c7892d
paper_authors.avatarUrl: /avatars/806b7a842d260990731f954af73896a3.svg
paper_authors.fullname: Wang Gaige
paper_authors.type: user
paper_authors.user: WangGaige
paper_authors._id: 67e132d77d88e5241c019393
paper_authors.name: Zhonghua Jiang
paper_authors._id: 67e132d77d88e5241c019394
paper_authors.name: Tiansong Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-24T12:54:12.524Z
paper_authors._id: 649d55b91e9de7b743897ba6
paper_authors.avatarUrl: /avatars/c8d3be7df5e2f7a6ffda1adcae1e7aac.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: Tiansong
paper_authors._id: 67e132d77d88e5241c019395
paper_authors.name: Zhiwen Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-24T12:54:25.569Z
paper_authors._id: 6729d2b802305fd2366b30a0
paper_authors.avatarUrl: /avatars/e5be25eecf5694e3b63a20c89a20653b.svg
paper_authors.fullname: Zhiwen Chen
paper_authors.type: user
paper_authors.user: chenzhiwen
paper_authors._id: 67e132d77d88e5241c019396
paper_authors.name: Chengfei Lv
paper_title: TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented
    Reality via 3D Gaussian Splatting
paper_summary: Realistic 3D full-body talking avatars hold great potential in AR, with
  applications ranging from e-commerce live streaming to holographic
  communication. Despite advances in 3D Gaussian Splatting (3DGS) for lifelike
  avatar creation, existing methods struggle with fine-grained control of facial
  expressions and body movements in full-body talking tasks. Additionally, they
  often lack sufficient details and cannot run in real-time on mobile devices. We
  present TaoAvatar, a high-fidelity, lightweight, 3DGS-based full-body talking
  avatar driven by various signals. Our approach starts by creating a
  personalized clothed human parametric template that binds Gaussians to
  represent appearances. We then pre-train a StyleUnet-based network to handle
  complex pose-dependent non-rigid deformation, which can capture high-frequency
  appearance details but is too resource-intensive for mobile devices. To
  overcome this, we "bake" the non-rigid deformations into a lightweight
  MLP-based network using a distillation technique and develop blend shapes to
  compensate for details. Extensive experiments show that TaoAvatar achieves
  state-of-the-art rendering quality while running in real-time across various
  devices, maintaining 90 FPS on high-definition stereo devices such as the Apple
  Vision Pro.
paper_ai_summary: TaoAvatar is a high-fidelity, lightweight 3DGS-based avatar that achieves real-time performance across devices while maintaining high rendering quality, leveraging distillation techniques for non-rigid deformation.
paper_ai_keywords: 3D Gaussian Splatting 3DGS StyleUnet non-rigid deformation ML MLP distillation blend shapes real-time performance high-definition stereo devices Apple Vision Pro