Relightable Full-Body Gaussian Codec Avatars
We propose Relightable Full-Body Gaussian Codec Avatars, a new approach for
modeling relightable full-body avatars with fine-grained details including face
and hands. The unique challenge for relighting full-body avatars lies in the
large deformations caused by body articulation and the resulting impact on
appearance caused by light transport. Changes in body pose can dramatically
change the orientation of body surfaces with respect to lights, resulting in
both local appearance changes due to changes in local light transport
functions, as well as non-local changes due to occlusion between body parts. To
address this, we decompose the light transport into local and non-local
effects. Local appearance changes are modeled using learnable zonal harmonics
for diffuse radiance transfer. Unlike spherical harmonics, zonal harmonics are
highly efficient to rotate under articulation. This allows us to learn diffuse
radiance transfer in a local coordinate frame, which disentangles the local
radiance transfer from the articulation of the body. To account for non-local
appearance changes, we introduce a shadow network that predicts shadows given
precomputed incoming irradiance on a base mesh. This facilitates the learning
of non-local shadowing between the body parts. Finally, we use a deferred
shading approach to model specular radiance transfer and better capture
reflections and highlights such as eye glints. We demonstrate that our approach
successfully models both the local and non-local light transport required for
relightable full-body avatars, with a superior generalization ability under
novel illumination conditions and unseen poses.
679704f72ec68b41932bf52f
Shaofei Wang
admin_assigned
2025-01-27T15:03:00.388Z
66d4a5c535eff7194df053a0
/avatars/4113570792db1d323d1c0b5beb98aa44.svg
shaofei wang
user
sfwang23
679704f72ec68b41932bf530
Tomas Simon
admin_assigned
2025-01-27T15:03:09.841Z
66c7459792e9f5b19f55cc56
/avatars/24cae9556ea07c47d65d500646bcfe85.svg
Tomas Simon
user
Tombo89
679704f72ec68b41932bf531
Igor Santesteban
679704f72ec68b41932bf532
Timur Bagautdinov
admin_assigned
2025-01-27T15:03:21.527Z
63be0fe19a15a3e9419d3d7d
/avatars/145760938083ec031e30d20d321d9185.svg
Timur Bagautdinov
user
psycharo
679704f72ec68b41932bf533
Junxuan Li
679704f72ec68b41932bf534
Vasu Agrawal
admin_assigned
2025-01-27T15:03:35.858Z
6527fa0f763bcce52dfb1af7
/avatars/79bdaa7f1948dfec4d9b6139952fbfe1.svg
Vasu Agrawal
user
vasuagrawal4
679704f72ec68b41932bf535
Fabian Prada
679704f72ec68b41932bf536
Shoou-I Yu
679704f72ec68b41932bf537
Pace Nalbone
679704f72ec68b41932bf538
Matt Gramlich
679704f72ec68b41932bf539
Roman Lubachersky
679704f72ec68b41932bf53a
Chenglei Wu
679704f72ec68b41932bf53b
Javier Romero
679704f72ec68b41932bf53c
Jason Saragih
679704f72ec68b41932bf53d
Michael Zollhoefer
679704f72ec68b41932bf53e
Andreas Geiger
admin_assigned
2025-01-27T15:04:34.554Z
620cae049086f3c07f01e3d5
https://cdn-avatars.huggingface.co/v1/production/uploads/1644998139538-noauth.jpeg
Andreas Geiger
user
andreas-geiger
679704f72ec68b41932bf53f
Siyu Tang
679704f72ec68b41932bf540
Shunsuke Saito
extracted_pending
2025-01-27T04:01:00.985Z
630688edd37ce67e0e50e548
/avatars/1681e465d7649f67f94e1b69d236cb1e.svg
Shunsuke Saito
user
psyth
Relightable Full-Body Gaussian Codec Avatars
We propose Relightable Full-Body Gaussian Codec Avatars, a new approach for
modeling relightable full-body avatars with fine-grained details including face
and hands. The unique challenge for relighting full-body avatars lies in the
large deformations caused by body articulation and the resulting impact on
appearance caused by light transport. Changes in body pose can dramatically
change the orientation of body surfaces with respect to lights, resulting in
both local appearance changes due to changes in local light transport
functions, as well as non-local changes due to occlusion between body parts. To
address this, we decompose the light transport into local and non-local
effects. Local appearance changes are modeled using learnable zonal harmonics
for diffuse radiance transfer. Unlike spherical harmonics, zonal harmonics are
highly efficient to rotate under articulation. This allows us to learn diffuse
radiance transfer in a local coordinate frame, which disentangles the local
radiance transfer from the articulation of the body. To account for non-local
appearance changes, we introduce a shadow network that predicts shadows given
precomputed incoming irradiance on a base mesh. This facilitates the learning
of non-local shadowing between the body parts. Finally, we use a deferred
shading approach to model specular radiance transfer and better capture
reflections and highlights such as eye glints. We demonstrate that our approach
successfully models both the local and non-local light transport required for
relightable full-body avatars, with a superior generalization ability under
novel illumination conditions and unseen poses.
A new approach for relightable full-body avatars uses zonal harmonics for local light transport and a shadow network for non-local effects, enabling superior generalization under novel lighting and poses.
Relightable Full-Body Gaussian Codec Avatars zonal harmonics diffuse radiance transfer shadow network local light transport non-local appearance changes deferred shading specular radiance transfer relightable avatars body articulation light transport incoming irradiance base mesh eye glints