title: Expressive Whole-Body 3D Gaussian Avatar
summary: Facial expression and hand motions are necessary to express our emotions and
  interact with the world. Nevertheless, most of the 3D human avatars modeled
  from a casually captured video only support body motions without facial
  expressions and hand motions.In this work, we present ExAvatar, an expressive
  whole-body 3D human avatar learned from a short monocular video. We design
  ExAvatar as a combination of the whole-body parametric mesh model (SMPL-X) and
  3D Gaussian Splatting (3DGS). The main challenges are 1) a limited diversity of
  facial expressions and poses in the video and 2) the absence of 3D
  observations, such as 3D scans and RGBD images. The limited diversity in the
  video makes animations with novel facial expressions and poses non-trivial. In
  addition, the absence of 3D observations could cause significant ambiguity in
  human parts that are not observed in the video, which can result in noticeable
  artifacts under novel motions. To address them, we introduce our hybrid
  representation of the mesh and 3D Gaussians. Our hybrid representation treats
  each 3D Gaussian as a vertex on the surface with pre-defined connectivity
  information (i.e., triangle faces) between them following the mesh topology of
  SMPL-X. It makes our ExAvatar animatable with novel facial expressions by
  driven by the facial expression space of SMPL-X. In addition, by using
  connectivity-based regularizers, we significantly reduce artifacts in novel
  facial expressions and poses.
paper_authors._id: 66aaf581e7af78c1374e4cc5
paper_authors.name: Gyeongsik Moon
paper_authors._id: 66aaf581e7af78c1374e4cc6
paper_authors.name: Takaaki Shiratori
paper_authors._id: 66aaf581e7af78c1374e4cc7
paper_authors.name: Shunsuke Saito
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T15:46:59.076Z
paper_authors._id: 630688edd37ce67e0e50e548
paper_authors.avatarUrl: /avatars/1681e465d7649f67f94e1b69d236cb1e.svg
paper_authors.fullname: Shunsuke Saito
paper_authors.type: user
paper_authors.user: psyth
paper_title: Expressive Whole-Body 3D Gaussian Avatar
paper_summary: Facial expression and hand motions are necessary to express our emotions and
  interact with the world. Nevertheless, most of the 3D human avatars modeled
  from a casually captured video only support body motions without facial
  expressions and hand motions.In this work, we present ExAvatar, an expressive
  whole-body 3D human avatar learned from a short monocular video. We design
  ExAvatar as a combination of the whole-body parametric mesh model (SMPL-X) and
  3D Gaussian Splatting (3DGS). The main challenges are 1) a limited diversity of
  facial expressions and poses in the video and 2) the absence of 3D
  observations, such as 3D scans and RGBD images. The limited diversity in the
  video makes animations with novel facial expressions and poses non-trivial. In
  addition, the absence of 3D observations could cause significant ambiguity in
  human parts that are not observed in the video, which can result in noticeable
  artifacts under novel motions. To address them, we introduce our hybrid
  representation of the mesh and 3D Gaussians. Our hybrid representation treats
  each 3D Gaussian as a vertex on the surface with pre-defined connectivity
  information (i.e., triangle faces) between them following the mesh topology of
  SMPL-X. It makes our ExAvatar animatable with novel facial expressions by
  driven by the facial expression space of SMPL-X. In addition, by using
  connectivity-based regularizers, we significantly reduce artifacts in novel
  facial expressions and poses.
paper_ai_keywords: whole-body parametric mesh model (SMPL-X) 3D Gaussian Splatting (3DGS) hybrid representation 3D Gaussians mesh topology facial expression space connectivity-based regularizers