title: MixRT: Mixed Neural Representations For Real-Time NeRF Rendering
summary: Neural Radiance Field (NeRF) has emerged as a leading technique for novel
  view synthesis, owing to its impressive photorealistic reconstruction and
  rendering capability. Nevertheless, achieving real-time NeRF rendering in
  large-scale scenes has presented challenges, often leading to the adoption of
  either intricate baked mesh representations with a substantial number of
  triangles or resource-intensive ray marching in baked representations. We
  challenge these conventions, observing that high-quality geometry, represented
  by meshes with substantial triangles, is not necessary for achieving
  photorealistic rendering quality. Consequently, we propose MixRT, a novel NeRF
  representation that includes a low-quality mesh, a view-dependent displacement
  map, and a compressed NeRF model. This design effectively harnesses the
  capabilities of existing graphics hardware, thus enabling real-time NeRF
  rendering on edge devices. Leveraging a highly-optimized WebGL-based rendering
  framework, our proposed MixRT attains real-time rendering speeds on edge
  devices (over 30 FPS at a resolution of 1280 x 720 on a MacBook M1 Pro laptop),
  better rendering quality (0.2 PSNR higher in indoor scenes of the Unbounded-360
  datasets), and a smaller storage size (less than 80% compared to
  state-of-the-art methods).
paper_authors._id: 658261522db4c45ef3f710f1
paper_authors.name: Chaojian Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:21:59.984Z
paper_authors._id: 64f1ec9e5894f48e7270dacd
paper_authors.avatarUrl: /avatars/5aca8dfec4a31d49984dfec23541f067.svg
paper_authors.fullname: Li
paper_authors.type: user
paper_authors.user: Chaojian
paper_authors._id: 658261522db4c45ef3f710f2
paper_authors.name: Bichen Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:22:21.914Z
paper_authors._id: 6363749520c2cf37bccd9e84
paper_authors.avatarUrl: /avatars/e9de7911e67090d883f15a7a58f18e4e.svg
paper_authors.fullname: Bichen Wu
paper_authors.type: user
paper_authors.user: bichenwu
paper_authors._id: 658261522db4c45ef3f710f3
paper_authors.name: Peter Vajda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:22:28.593Z
paper_authors._id: 636f4b35af79dd193603342b
paper_authors.avatarUrl: /avatars/577328195738d92ac41a3d0ff3108d7c.svg
paper_authors.fullname: Peter Vajda
paper_authors.type: user
paper_authors.user: vaphab
paper_authors._id: 658261522db4c45ef3f710f4
paper_authors.name: Yingyan
paper_authors._id: 658261522db4c45ef3f710f5
paper_authors.name: Lin
paper_title: MixRT: Mixed Neural Representations For Real-Time NeRF Rendering
paper_summary: Neural Radiance Field (NeRF) has emerged as a leading technique for novel
  view synthesis, owing to its impressive photorealistic reconstruction and
  rendering capability. Nevertheless, achieving real-time NeRF rendering in
  large-scale scenes has presented challenges, often leading to the adoption of
  either intricate baked mesh representations with a substantial number of
  triangles or resource-intensive ray marching in baked representations. We
  challenge these conventions, observing that high-quality geometry, represented
  by meshes with substantial triangles, is not necessary for achieving
  photorealistic rendering quality. Consequently, we propose MixRT, a novel NeRF
  representation that includes a low-quality mesh, a view-dependent displacement
  map, and a compressed NeRF model. This design effectively harnesses the
  capabilities of existing graphics hardware, thus enabling real-time NeRF
  rendering on edge devices. Leveraging a highly-optimized WebGL-based rendering
  framework, our proposed MixRT attains real-time rendering speeds on edge
  devices (over 30 FPS at a resolution of 1280 x 720 on a MacBook M1 Pro laptop),
  better rendering quality (0.2 PSNR higher in indoor scenes of the Unbounded-360
  datasets), and a smaller storage size (less than 80% compared to
  state-of-the-art methods).
paper_ai_keywords: Neural Radiance Field (NeRF) novel view synthesis photorealistic reconstruction real-time rendering large-scale scenes baked mesh representations ray marching high-quality geometry low-quality mesh view-dependent displacement map compressed NeRF model WebGL-based rendering framework