Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D
  Scene Representations
Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations,
capable of high quality novel view synthesis of complex scenes. While NeRFs
have been applied to graphics, vision, and robotics, problems with slow
rendering speed and characteristic visual artifacts prevent adoption in many
use cases. In this work, we investigate combining an autoencoder (AE) with a
NeRF, in which latent features (instead of colours) are rendered and then
convolutionally decoded. The resulting latent-space NeRF can produce novel
views with higher quality than standard colour-space NeRFs, as the AE can
correct certain visual artifacts, while rendering over three times faster. Our
work is orthogonal to other techniques for improving NeRF efficiency. Further,
we can control the tradeoff between efficiency and image quality by shrinking
the AE architecture, achieving over 13 times faster rendering with only a small
drop in performance. We hope that our approach can form the basis of an
efficient, yet high-fidelity, 3D scene representation for downstream tasks,
especially when retaining differentiability is useful, as in many robotics
scenarios requiring continual learning.
653f2269eefdcdc9fdb301d9
Tristan Aumentado-Armstrong
admin_assigned
2023-10-30T14:05:26.567Z
637eca3e1618112bdd7f6807
/avatars/04a3809542241f5315654153dc7b4375.svg
Tristan Aumentado-Armstrong
user
taumarm
653f2269eefdcdc9fdb301da
Ashkan Mirzaei
admin_assigned
2023-10-30T14:05:38.174Z
634db15dd00bb5d92c3bd94f
/avatars/9a6a1231bc5205911272d83527593f1a.svg
Ashkan Mirzaei
user
ashmrz
653f2269eefdcdc9fdb301db
Marcus A. Brubaker
653f2269eefdcdc9fdb301dc
Jonathan Kelly
admin_assigned
2023-10-30T14:05:53.563Z
64499189d2a85ea9466ab803
/avatars/fc90bf3a1a5c896afdfefc3d1932d33a.svg
Jonathan Kelly
user
Thatguyjk723
653f2269eefdcdc9fdb301dd
Alex Levinshtein
653f2269eefdcdc9fdb301de
Konstantinos G. Derpanis
653f2269eefdcdc9fdb301df
Igor Gilitschenski
admin_assigned
2023-10-30T14:06:17.824Z
64893f0b9ab28735f1a1ec0b
https://cdn-avatars.huggingface.co/v1/production/uploads/64893f0b9ab28735f1a1ec0b/2lPBjV16l2qLEWOKU2zwR.jpeg
Igor Gilitschenski
user
igilitschenski
Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D
  Scene Representations
Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations,
capable of high quality novel view synthesis of complex scenes. While NeRFs
have been applied to graphics, vision, and robotics, problems with slow
rendering speed and characteristic visual artifacts prevent adoption in many
use cases. In this work, we investigate combining an autoencoder (AE) with a
NeRF, in which latent features (instead of colours) are rendered and then
convolutionally decoded. The resulting latent-space NeRF can produce novel
views with higher quality than standard colour-space NeRFs, as the AE can
correct certain visual artifacts, while rendering over three times faster. Our
work is orthogonal to other techniques for improving NeRF efficiency. Further,
we can control the tradeoff between efficiency and image quality by shrinking
the AE architecture, achieving over 13 times faster rendering with only a small
drop in performance. We hope that our approach can form the basis of an
efficient, yet high-fidelity, 3D scene representation for downstream tasks,
especially when retaining differentiability is useful, as in many robotics
scenarios requiring continual learning.
Combining autoencoders with Neural Radiance Fields improves rendering speed and image quality, enabling efficient and high-fidelity 3D scene representation suitable for robotics.
Neural Radiance Fields NeRFs autoencoder AE latent features convolutional decoding latent-space NeRF visual artifacts novel view synthesis differentiability continual learning