title: HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian
    Splatting
summary: We have recently seen tremendous progress in photo-real human modeling and
  rendering. Yet, efficiently rendering realistic human performance and
  integrating it into the rasterization pipeline remains challenging. In this
  paper, we present HiFi4G, an explicit and compact Gaussian-based approach for
  high-fidelity human performance rendering from dense footage. Our core
  intuition is to marry the 3D Gaussian representation with non-rigid tracking,
  achieving a compact and compression-friendly representation. We first propose a
  dual-graph mechanism to obtain motion priors, with a coarse deformation graph
  for effective initialization and a fine-grained Gaussian graph to enforce
  subsequent constraints. Then, we utilize a 4D Gaussian optimization scheme with
  adaptive spatial-temporal regularizers to effectively balance the non-rigid
  prior and Gaussian updating. We also present a companion compression scheme
  with residual compensation for immersive experiences on various platforms. It
  achieves a substantial compression rate of approximately 25 times, with less
  than 2MB of storage per frame. Extensive experiments demonstrate the
  effectiveness of our approach, which significantly outperforms existing
  approaches in terms of optimization speed, rendering quality, and storage
  overhead.
paper_authors._id: 65713bdf87a92b76924ecd4d
paper_authors.name: Yuheng Jiang
paper_authors._id: 65713bdf87a92b76924ecd4e
paper_authors.name: Zhehao Shen
paper_authors._id: 65713bdf87a92b76924ecd4f
paper_authors.name: Penghao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T11:56:28.616Z
paper_authors._id: 654a064a8ceb5deea8c53acb
paper_authors.avatarUrl: /avatars/b6fc5c4f62c75b0866926a0da541620f.svg
paper_authors.fullname: Penghao Wang
paper_authors.type: user
paper_authors.user: phw00007
paper_authors._id: 65713bdf87a92b76924ecd50
paper_authors.name: Zhuo Su
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T11:56:35.247Z
paper_authors._id: 655238b2957bf8a398eca2a1
paper_authors.avatarUrl: /avatars/f0e5af3de113b998bcf84953ecd8b930.svg
paper_authors.fullname: Zhuo Su
paper_authors.type: user
paper_authors.user: hellozhuo
paper_authors._id: 65713bdf87a92b76924ecd51
paper_authors.name: Yu Hong
paper_authors._id: 65713bdf87a92b76924ecd52
paper_authors.name: Yingliang Zhang
paper_authors._id: 65713bdf87a92b76924ecd53
paper_authors.name: Jingyi Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T11:57:01.625Z
paper_authors._id: 63ffcec6e7767a89533b8f80
paper_authors.avatarUrl: /avatars/b9e8673943741c2e7b80b15a061bb951.svg
paper_authors.fullname: JINGYI Yu
paper_authors.type: user
paper_authors.user: Daluuu
paper_authors._id: 65713bdf87a92b76924ecd54
paper_authors.name: Lan Xu
paper_title: HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian
    Splatting
paper_summary: We have recently seen tremendous progress in photo-real human modeling and
  rendering. Yet, efficiently rendering realistic human performance and
  integrating it into the rasterization pipeline remains challenging. In this
  paper, we present HiFi4G, an explicit and compact Gaussian-based approach for
  high-fidelity human performance rendering from dense footage. Our core
  intuition is to marry the 3D Gaussian representation with non-rigid tracking,
  achieving a compact and compression-friendly representation. We first propose a
  dual-graph mechanism to obtain motion priors, with a coarse deformation graph
  for effective initialization and a fine-grained Gaussian graph to enforce
  subsequent constraints. Then, we utilize a 4D Gaussian optimization scheme with
  adaptive spatial-temporal regularizers to effectively balance the non-rigid
  prior and Gaussian updating. We also present a companion compression scheme
  with residual compensation for immersive experiences on various platforms. It
  achieves a substantial compression rate of approximately 25 times, with less
  than 2MB of storage per frame. Extensive experiments demonstrate the
  effectiveness of our approach, which significantly outperforms existing
  approaches in terms of optimization speed, rendering quality, and storage
  overhead.
paper_ai_summary: HiFi4G uses a Gaussian-based approach with 3D Gaussian representation and non-rigid tracking for efficient high-fidelity human performance rendering, offering significant compression and quality improvements.
paper_ai_keywords: Gaussian-based approach 3D Gaussian representation non-rigid tracking dual-graph mechanism coarse deformation graph fine-grained Gaussian graph 4D Gaussian optimization adaptive spatial-temporal regularizers residual compensation