title: Relightable Full-Body Gaussian Codec Avatars
summary: We propose Relightable Full-Body Gaussian Codec Avatars, a new approach for
  modeling relightable full-body avatars with fine-grained details including face
  and hands. The unique challenge for relighting full-body avatars lies in the
  large deformations caused by body articulation and the resulting impact on
  appearance caused by light transport. Changes in body pose can dramatically
  change the orientation of body surfaces with respect to lights, resulting in
  both local appearance changes due to changes in local light transport
  functions, as well as non-local changes due to occlusion between body parts. To
  address this, we decompose the light transport into local and non-local
  effects. Local appearance changes are modeled using learnable zonal harmonics
  for diffuse radiance transfer. Unlike spherical harmonics, zonal harmonics are
  highly efficient to rotate under articulation. This allows us to learn diffuse
  radiance transfer in a local coordinate frame, which disentangles the local
  radiance transfer from the articulation of the body. To account for non-local
  appearance changes, we introduce a shadow network that predicts shadows given
  precomputed incoming irradiance on a base mesh. This facilitates the learning
  of non-local shadowing between the body parts. Finally, we use a deferred
  shading approach to model specular radiance transfer and better capture
  reflections and highlights such as eye glints. We demonstrate that our approach
  successfully models both the local and non-local light transport required for
  relightable full-body avatars, with a superior generalization ability under
  novel illumination conditions and unseen poses.
paper_authors._id: 679704f72ec68b41932bf52f
paper_authors.name: Shaofei Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T15:03:00.388Z
paper_authors._id: 66d4a5c535eff7194df053a0
paper_authors.avatarUrl: /avatars/4113570792db1d323d1c0b5beb98aa44.svg
paper_authors.fullname: shaofei wang
paper_authors.type: user
paper_authors.user: sfwang23
paper_authors._id: 679704f72ec68b41932bf530
paper_authors.name: Tomas Simon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T15:03:09.841Z
paper_authors._id: 66c7459792e9f5b19f55cc56
paper_authors.avatarUrl: /avatars/24cae9556ea07c47d65d500646bcfe85.svg
paper_authors.fullname: Tomas Simon
paper_authors.type: user
paper_authors.user: Tombo89
paper_authors._id: 679704f72ec68b41932bf531
paper_authors.name: Igor Santesteban
paper_authors._id: 679704f72ec68b41932bf532
paper_authors.name: Timur Bagautdinov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T15:03:21.527Z
paper_authors._id: 63be0fe19a15a3e9419d3d7d
paper_authors.avatarUrl: /avatars/145760938083ec031e30d20d321d9185.svg
paper_authors.fullname: Timur Bagautdinov
paper_authors.type: user
paper_authors.user: psycharo
paper_authors._id: 679704f72ec68b41932bf533
paper_authors.name: Junxuan Li
paper_authors._id: 679704f72ec68b41932bf534
paper_authors.name: Vasu Agrawal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T15:03:35.858Z
paper_authors._id: 6527fa0f763bcce52dfb1af7
paper_authors.avatarUrl: /avatars/79bdaa7f1948dfec4d9b6139952fbfe1.svg
paper_authors.fullname: Vasu Agrawal
paper_authors.type: user
paper_authors.user: vasuagrawal4
paper_authors._id: 679704f72ec68b41932bf535
paper_authors.name: Fabian Prada
paper_authors._id: 679704f72ec68b41932bf536
paper_authors.name: Shoou-I Yu
paper_authors._id: 679704f72ec68b41932bf537
paper_authors.name: Pace Nalbone
paper_authors._id: 679704f72ec68b41932bf538
paper_authors.name: Matt Gramlich
paper_authors._id: 679704f72ec68b41932bf539
paper_authors.name: Roman Lubachersky
paper_authors._id: 679704f72ec68b41932bf53a
paper_authors.name: Chenglei Wu
paper_authors._id: 679704f72ec68b41932bf53b
paper_authors.name: Javier Romero
paper_authors._id: 679704f72ec68b41932bf53c
paper_authors.name: Jason Saragih
paper_authors._id: 679704f72ec68b41932bf53d
paper_authors.name: Michael Zollhoefer
paper_authors._id: 679704f72ec68b41932bf53e
paper_authors.name: Andreas Geiger
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T15:04:34.554Z
paper_authors._id: 620cae049086f3c07f01e3d5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644998139538-noauth.jpeg
paper_authors.fullname: Andreas Geiger
paper_authors.type: user
paper_authors.user: andreas-geiger
paper_authors._id: 679704f72ec68b41932bf53f
paper_authors.name: Siyu Tang
paper_authors._id: 679704f72ec68b41932bf540
paper_authors.name: Shunsuke Saito
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-27T04:01:00.985Z
paper_authors._id: 630688edd37ce67e0e50e548
paper_authors.avatarUrl: /avatars/1681e465d7649f67f94e1b69d236cb1e.svg
paper_authors.fullname: Shunsuke Saito
paper_authors.type: user
paper_authors.user: psyth
paper_title: Relightable Full-Body Gaussian Codec Avatars
paper_summary: We propose Relightable Full-Body Gaussian Codec Avatars, a new approach for
  modeling relightable full-body avatars with fine-grained details including face
  and hands. The unique challenge for relighting full-body avatars lies in the
  large deformations caused by body articulation and the resulting impact on
  appearance caused by light transport. Changes in body pose can dramatically
  change the orientation of body surfaces with respect to lights, resulting in
  both local appearance changes due to changes in local light transport
  functions, as well as non-local changes due to occlusion between body parts. To
  address this, we decompose the light transport into local and non-local
  effects. Local appearance changes are modeled using learnable zonal harmonics
  for diffuse radiance transfer. Unlike spherical harmonics, zonal harmonics are
  highly efficient to rotate under articulation. This allows us to learn diffuse
  radiance transfer in a local coordinate frame, which disentangles the local
  radiance transfer from the articulation of the body. To account for non-local
  appearance changes, we introduce a shadow network that predicts shadows given
  precomputed incoming irradiance on a base mesh. This facilitates the learning
  of non-local shadowing between the body parts. Finally, we use a deferred
  shading approach to model specular radiance transfer and better capture
  reflections and highlights such as eye glints. We demonstrate that our approach
  successfully models both the local and non-local light transport required for
  relightable full-body avatars, with a superior generalization ability under
  novel illumination conditions and unseen poses.
paper_ai_keywords: Relightable Full-Body Gaussian Codec Avatars Gaussian Codec Avatars full-body avatars body articulation light transport learnable zonal harmonics diffuse radiance transfer spherical harmonics zonal harmonics local radiance transfer shadow network incoming irradiance deferred shading specular radiance transfer eye glints