title: Towards Practical Capture of High-Fidelity Relightable Avatars
summary: In this paper, we propose a novel framework, Tracking-free Relightable Avatar
  (TRAvatar), for capturing and reconstructing high-fidelity 3D avatars. Compared
  to previous methods, TRAvatar works in a more practical and efficient setting.
  Specifically, TRAvatar is trained with dynamic image sequences captured in a
  Light Stage under varying lighting conditions, enabling realistic relighting
  and real-time animation for avatars in diverse scenes. Additionally, TRAvatar
  allows for tracking-free avatar capture and obviates the need for accurate
  surface tracking under varying illumination conditions. Our contributions are
  two-fold: First, we propose a novel network architecture that explicitly builds
  on and ensures the satisfaction of the linear nature of lighting. Trained on
  simple group light captures, TRAvatar can predict the appearance in real-time
  with a single forward pass, achieving high-quality relighting effects under
  illuminations of arbitrary environment maps. Second, we jointly optimize the
  facial geometry and relightable appearance from scratch based on image
  sequences, where the tracking is implicitly learned. This tracking-free
  approach brings robustness for establishing temporal correspondences between
  frames under different lighting conditions. Extensive qualitative and
  quantitative experiments demonstrate that our framework achieves superior
  performance for photorealistic avatar animation and relighting.
paper_authors._id: 64fe842852992431a6478a99
paper_authors.name: Haotian Yang
paper_authors._id: 64fe842852992431a6478a9a
paper_authors.name: Mingwu Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-11T11:23:52.527Z
paper_authors._id: 647dcab4becb41a272921250
paper_authors.avatarUrl: /avatars/1e4fa540109c4963f54ccad9bd9e9b42.svg
paper_authors.fullname: Mingwu Zheng
paper_authors.type: user
paper_authors.user: MuonZ
paper_authors._id: 64fe842852992431a6478a9b
paper_authors.name: Wanquan Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-11T11:24:00.791Z
paper_authors._id: 64462aec86679b753ff7ece0
paper_authors.avatarUrl: /avatars/ff0e1e693b984c9d569dc462fa7fe4ef.svg
paper_authors.fullname: Wanquan Feng
paper_authors.type: user
paper_authors.user: wanquan
paper_authors._id: 64fe842852992431a6478a9c
paper_authors.name: Haibin Huang
paper_authors._id: 64fe842852992431a6478a9d
paper_authors.name: Yu-Kun Lai
paper_authors._id: 64fe842852992431a6478a9e
paper_authors.name: Pengfei Wan
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-18T04:41:53.913Z
paper_authors._id: 662f93942510ef5735d7ad00
paper_authors.avatarUrl: /avatars/dc9486db75869ce902d0a638eea126bd.svg
paper_authors.fullname: magicwpf
paper_authors.type: user
paper_authors.user: magicwpf
paper_authors._id: 64fe842852992431a6478a9f
paper_authors.name: Zhongyuan Wang
paper_authors._id: 64fe842852992431a6478aa0
paper_authors.name: Chongyang Ma
paper_title: Towards Practical Capture of High-Fidelity Relightable Avatars
paper_summary: In this paper, we propose a novel framework, Tracking-free Relightable Avatar
  (TRAvatar), for capturing and reconstructing high-fidelity 3D avatars. Compared
  to previous methods, TRAvatar works in a more practical and efficient setting.
  Specifically, TRAvatar is trained with dynamic image sequences captured in a
  Light Stage under varying lighting conditions, enabling realistic relighting
  and real-time animation for avatars in diverse scenes. Additionally, TRAvatar
  allows for tracking-free avatar capture and obviates the need for accurate
  surface tracking under varying illumination conditions. Our contributions are
  two-fold: First, we propose a novel network architecture that explicitly builds
  on and ensures the satisfaction of the linear nature of lighting. Trained on
  simple group light captures, TRAvatar can predict the appearance in real-time
  with a single forward pass, achieving high-quality relighting effects under
  illuminations of arbitrary environment maps. Second, we jointly optimize the
  facial geometry and relightable appearance from scratch based on image
  sequences, where the tracking is implicitly learned. This tracking-free
  approach brings robustness for establishing temporal correspondences between
  frames under different lighting conditions. Extensive qualitative and
  quantitative experiments demonstrate that our framework achieves superior
  performance for photorealistic avatar animation and relighting.
paper_ai_keywords: TRAvatar Light Stage real-time animation relighting 3D avatars tracking-free network architecture linear nature of lighting group light captures real-time prediction semantic maps facial geometry temporal correspondences