Expressive Whole-Body 3D Gaussian Avatar
Facial expression and hand motions are necessary to express our emotions and
interact with the world. Nevertheless, most of the 3D human avatars modeled
from a casually captured video only support body motions without facial
expressions and hand motions.In this work, we present ExAvatar, an expressive
whole-body 3D human avatar learned from a short monocular video. We design
ExAvatar as a combination of the whole-body parametric mesh model (SMPL-X) and
3D Gaussian Splatting (3DGS). The main challenges are 1) a limited diversity of
facial expressions and poses in the video and 2) the absence of 3D
observations, such as 3D scans and RGBD images. The limited diversity in the
video makes animations with novel facial expressions and poses non-trivial. In
addition, the absence of 3D observations could cause significant ambiguity in
human parts that are not observed in the video, which can result in noticeable
artifacts under novel motions. To address them, we introduce our hybrid
representation of the mesh and 3D Gaussians. Our hybrid representation treats
each 3D Gaussian as a vertex on the surface with pre-defined connectivity
information (i.e., triangle faces) between them following the mesh topology of
SMPL-X. It makes our ExAvatar animatable with novel facial expressions by
driven by the facial expression space of SMPL-X. In addition, by using
connectivity-based regularizers, we significantly reduce artifacts in novel
facial expressions and poses.
66aaf581e7af78c1374e4cc5
Gyeongsik Moon
66aaf581e7af78c1374e4cc6
Takaaki Shiratori
66aaf581e7af78c1374e4cc7
Shunsuke Saito
admin_assigned
2024-08-02T15:46:59.076Z
630688edd37ce67e0e50e548
/avatars/1681e465d7649f67f94e1b69d236cb1e.svg
Shunsuke Saito
user
psyth
Expressive Whole-Body 3D Gaussian Avatar
Facial expression and hand motions are necessary to express our emotions and
interact with the world. Nevertheless, most of the 3D human avatars modeled
from a casually captured video only support body motions without facial
expressions and hand motions.In this work, we present ExAvatar, an expressive
whole-body 3D human avatar learned from a short monocular video. We design
ExAvatar as a combination of the whole-body parametric mesh model (SMPL-X) and
3D Gaussian Splatting (3DGS). The main challenges are 1) a limited diversity of
facial expressions and poses in the video and 2) the absence of 3D
observations, such as 3D scans and RGBD images. The limited diversity in the
video makes animations with novel facial expressions and poses non-trivial. In
addition, the absence of 3D observations could cause significant ambiguity in
human parts that are not observed in the video, which can result in noticeable
artifacts under novel motions. To address them, we introduce our hybrid
representation of the mesh and 3D Gaussians. Our hybrid representation treats
each 3D Gaussian as a vertex on the surface with pre-defined connectivity
information (i.e., triangle faces) between them following the mesh topology of
SMPL-X. It makes our ExAvatar animatable with novel facial expressions by
driven by the facial expression space of SMPL-X. In addition, by using
connectivity-based regularizers, we significantly reduce artifacts in novel
facial expressions and poses.
ExAvatar is a 3D human avatar combining SMPL-X mesh and 3D Gaussian Splatting, learned from a single monocular video, enabling expressive animations with novel facial expressions and poses.
ExAvatar SMPL-X 3D Gaussian Splatting 3DGS face expressions poses hybrid representation connectivity-based regularizers novel animations