Towards Practical Capture of High-Fidelity Relightable Avatars
In this paper, we propose a novel framework, Tracking-free Relightable Avatar
(TRAvatar), for capturing and reconstructing high-fidelity 3D avatars. Compared
to previous methods, TRAvatar works in a more practical and efficient setting.
Specifically, TRAvatar is trained with dynamic image sequences captured in a
Light Stage under varying lighting conditions, enabling realistic relighting
and real-time animation for avatars in diverse scenes. Additionally, TRAvatar
allows for tracking-free avatar capture and obviates the need for accurate
surface tracking under varying illumination conditions. Our contributions are
two-fold: First, we propose a novel network architecture that explicitly builds
on and ensures the satisfaction of the linear nature of lighting. Trained on
simple group light captures, TRAvatar can predict the appearance in real-time
with a single forward pass, achieving high-quality relighting effects under
illuminations of arbitrary environment maps. Second, we jointly optimize the
facial geometry and relightable appearance from scratch based on image
sequences, where the tracking is implicitly learned. This tracking-free
approach brings robustness for establishing temporal correspondences between
frames under different lighting conditions. Extensive qualitative and
quantitative experiments demonstrate that our framework achieves superior
performance for photorealistic avatar animation and relighting.
64fe842852992431a6478a99
Haotian Yang
64fe842852992431a6478a9a
Mingwu Zheng
admin_assigned
2023-09-11T11:23:52.527Z
647dcab4becb41a272921250
/avatars/1e4fa540109c4963f54ccad9bd9e9b42.svg
Mingwu Zheng
user
MuonZ
64fe842852992431a6478a9b
Wanquan Feng
admin_assigned
2023-09-11T11:24:00.791Z
64462aec86679b753ff7ece0
/avatars/ff0e1e693b984c9d569dc462fa7fe4ef.svg
Wanquan Feng
user
wanquan
64fe842852992431a6478a9c
Haibin Huang
64fe842852992431a6478a9d
Yu-Kun Lai
64fe842852992431a6478a9e
Pengfei Wan
extracted_confirmed
2025-03-18T04:41:53.913Z
662f93942510ef5735d7ad00
https://cdn-avatars.huggingface.co/v1/production/uploads/662f93942510ef5735d7ad00/ZIDIPm63sncIHFTT5b0uR.png
magicwpf
user
magicwpf
64fe842852992431a6478a9f
Zhongyuan Wang
64fe842852992431a6478aa0
Chongyang Ma
Towards Practical Capture of High-Fidelity Relightable Avatars
In this paper, we propose a novel framework, Tracking-free Relightable Avatar
(TRAvatar), for capturing and reconstructing high-fidelity 3D avatars. Compared
to previous methods, TRAvatar works in a more practical and efficient setting.
Specifically, TRAvatar is trained with dynamic image sequences captured in a
Light Stage under varying lighting conditions, enabling realistic relighting
and real-time animation for avatars in diverse scenes. Additionally, TRAvatar
allows for tracking-free avatar capture and obviates the need for accurate
surface tracking under varying illumination conditions. Our contributions are
two-fold: First, we propose a novel network architecture that explicitly builds
on and ensures the satisfaction of the linear nature of lighting. Trained on
simple group light captures, TRAvatar can predict the appearance in real-time
with a single forward pass, achieving high-quality relighting effects under
illuminations of arbitrary environment maps. Second, we jointly optimize the
facial geometry and relightable appearance from scratch based on image
sequences, where the tracking is implicitly learned. This tracking-free
approach brings robustness for establishing temporal correspondences between
frames under different lighting conditions. Extensive qualitative and
quantitative experiments demonstrate that our framework achieves superior
performance for photorealistic avatar animation and relighting.
A tracking-free framework captures and reconstructs high-fidelity 3D avatars with realistic relighting and real-time animation using a novel network architecture optimized for capturing dynamic lighting conditions.
Tracking-free avatar capture Light Stage dynamic image sequences real-time animation network architecture lighting linear nature group light captures real-time relighting environment maps temporal correspondences photorealistic avatar animation