MixRT: Mixed Neural Representations For Real-Time NeRF Rendering
Neural Radiance Field (NeRF) has emerged as a leading technique for novel
view synthesis, owing to its impressive photorealistic reconstruction and
rendering capability. Nevertheless, achieving real-time NeRF rendering in
large-scale scenes has presented challenges, often leading to the adoption of
either intricate baked mesh representations with a substantial number of
triangles or resource-intensive ray marching in baked representations. We
challenge these conventions, observing that high-quality geometry, represented
by meshes with substantial triangles, is not necessary for achieving
photorealistic rendering quality. Consequently, we propose MixRT, a novel NeRF
representation that includes a low-quality mesh, a view-dependent displacement
map, and a compressed NeRF model. This design effectively harnesses the
capabilities of existing graphics hardware, thus enabling real-time NeRF
rendering on edge devices. Leveraging a highly-optimized WebGL-based rendering
framework, our proposed MixRT attains real-time rendering speeds on edge
devices (over 30 FPS at a resolution of 1280 x 720 on a MacBook M1 Pro laptop),
better rendering quality (0.2 PSNR higher in indoor scenes of the Unbounded-360
datasets), and a smaller storage size (less than 80% compared to
state-of-the-art methods).
658261522db4c45ef3f710f1
Chaojian Li
admin_assigned
2023-12-20T09:21:59.984Z
64f1ec9e5894f48e7270dacd
/avatars/5aca8dfec4a31d49984dfec23541f067.svg
Li
user
Chaojian
658261522db4c45ef3f710f2
Bichen Wu
admin_assigned
2023-12-20T09:22:21.914Z
6363749520c2cf37bccd9e84
/avatars/e9de7911e67090d883f15a7a58f18e4e.svg
Bichen Wu
user
bichenwu
658261522db4c45ef3f710f3
Peter Vajda
admin_assigned
2023-12-20T09:22:28.593Z
636f4b35af79dd193603342b
/avatars/577328195738d92ac41a3d0ff3108d7c.svg
Peter Vajda
user
vaphab
658261522db4c45ef3f710f4
Yingyan
658261522db4c45ef3f710f5
Lin
MixRT: Mixed Neural Representations For Real-Time NeRF Rendering
Neural Radiance Field (NeRF) has emerged as a leading technique for novel
view synthesis, owing to its impressive photorealistic reconstruction and
rendering capability. Nevertheless, achieving real-time NeRF rendering in
large-scale scenes has presented challenges, often leading to the adoption of
either intricate baked mesh representations with a substantial number of
triangles or resource-intensive ray marching in baked representations. We
challenge these conventions, observing that high-quality geometry, represented
by meshes with substantial triangles, is not necessary for achieving
photorealistic rendering quality. Consequently, we propose MixRT, a novel NeRF
representation that includes a low-quality mesh, a view-dependent displacement
map, and a compressed NeRF model. This design effectively harnesses the
capabilities of existing graphics hardware, thus enabling real-time NeRF
rendering on edge devices. Leveraging a highly-optimized WebGL-based rendering
framework, our proposed MixRT attains real-time rendering speeds on edge
devices (over 30 FPS at a resolution of 1280 x 720 on a MacBook M1 Pro laptop),
better rendering quality (0.2 PSNR higher in indoor scenes of the Unbounded-360
datasets), and a smaller storage size (less than 80% compared to
state-of-the-art methods).
MixRT, a novel NeRF representation combining a low-quality mesh, view-dependent displacement map, and compressed NeRF model, achieves real-time rendering on edge devices with improved quality and reduced storage.
Neural Radiance Field NeRF novel view synthesis real-time rendering large-scale scenes ray marching baked mesh representations view-dependent displacement map compressed NeRF model WebGL rendering quality Unbounded-360 datasets PSNR storage size