title: NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads
summary: We focus on reconstructing high-fidelity radiance fields of human heads,
  capturing their animations over time, and synthesizing re-renderings from novel
  viewpoints at arbitrary time steps. To this end, we propose a new multi-view
  capture setup composed of 16 calibrated machine vision cameras that record
  time-synchronized images at 7.1 MP resolution and 73 frames per second. With
  our setup, we collect a new dataset of over 4700 high-resolution,
  high-framerate sequences of more than 220 human heads, from which we introduce
  a new human head reconstruction benchmark. The recorded sequences cover a wide
  range of facial dynamics, including head motions, natural expressions,
  emotions, and spoken language. In order to reconstruct high-fidelity human
  heads, we propose Dynamic Neural Radiance Fields using Hash Ensembles
  (NeRSemble). We represent scene dynamics by combining a deformation field and
  an ensemble of 3D multi-resolution hash encodings. The deformation field allows
  for precise modeling of simple scene movements, while the ensemble of hash
  encodings helps to represent complex dynamics. As a result, we obtain radiance
  field representations of human heads that capture motion over time and
  facilitate re-rendering of arbitrary novel viewpoints. In a series of
  experiments, we explore the design choices of our method and demonstrate that
  our approach outperforms state-of-the-art dynamic radiance field approaches by
  a significant margin.
paper_authors._id: 645483faec40bbbd012556c6
paper_authors.name: Tobias Kirschstein
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-07T19:39:30.809Z
paper_authors._id: 64806288cacb1c4a06946084
paper_authors.avatarUrl: /avatars/5101d2234279257d534d9e9d2f69527f.svg
paper_authors.fullname: Tobias Kirschstein
paper_authors.type: user
paper_authors.user: tobias-kirschstein
paper_authors._id: 645483faec40bbbd012556c7
paper_authors.name: Shenhan Qian
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-11-04T13:22:30.403Z
paper_authors._id: 6728ca5827fa1635d7b168aa
paper_authors.avatarUrl: /avatars/39d4bf9539d4b066aa5a553338d9e31e.svg
paper_authors.fullname: Shenhan Qian
paper_authors.type: user
paper_authors.user: Shenhan
paper_authors._id: 645483faec40bbbd012556c8
paper_authors.name: Simon Giebenhain
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-24T13:32:33.700Z
paper_authors._id: 64dc981b96fbcd6b19f687b5
paper_authors.avatarUrl: /avatars/6eca33e3fcca750d4f6e9b0f7d724d8e.svg
paper_authors.fullname: Giebenhain
paper_authors.type: user
paper_authors.user: SimonGiebenhain
paper_authors._id: 645483faec40bbbd012556c9
paper_authors.name: Tim Walter
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-05-25T17:14:56.136Z
paper_authors._id: 646f9768cc049b49e35e8147
paper_authors.avatarUrl: /avatars/326ffea9f66112d547c6f77d2f505802.svg
paper_authors.fullname: Tim Walter
paper_authors.type: user
paper_authors.user: TimWalter
paper_authors._id: 645483faec40bbbd012556ca
paper_authors.name: Matthias Nießner
paper_title: NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads
paper_summary: We focus on reconstructing high-fidelity radiance fields of human heads,
  capturing their animations over time, and synthesizing re-renderings from novel
  viewpoints at arbitrary time steps. To this end, we propose a new multi-view
  capture setup composed of 16 calibrated machine vision cameras that record
  time-synchronized images at 7.1 MP resolution and 73 frames per second. With
  our setup, we collect a new dataset of over 4700 high-resolution,
  high-framerate sequences of more than 220 human heads, from which we introduce
  a new human head reconstruction benchmark. The recorded sequences cover a wide
  range of facial dynamics, including head motions, natural expressions,
  emotions, and spoken language. In order to reconstruct high-fidelity human
  heads, we propose Dynamic Neural Radiance Fields using Hash Ensembles
  (NeRSemble). We represent scene dynamics by combining a deformation field and
  an ensemble of 3D multi-resolution hash encodings. The deformation field allows
  for precise modeling of simple scene movements, while the ensemble of hash
  encodings helps to represent complex dynamics. As a result, we obtain radiance
  field representations of human heads that capture motion over time and
  facilitate re-rendering of arbitrary novel viewpoints. In a series of
  experiments, we explore the design choices of our method and demonstrate that
  our approach outperforms state-of-the-art dynamic radiance field approaches by
  a significant margin.
paper_ai_keywords: Dynamic Neural Radiance Fields Hash Ensembles NeRSemble deformation field hash encodings radiance field representations