title: Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D
    Scene Representations
summary: Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations,
  capable of high quality novel view synthesis of complex scenes. While NeRFs
  have been applied to graphics, vision, and robotics, problems with slow
  rendering speed and characteristic visual artifacts prevent adoption in many
  use cases. In this work, we investigate combining an autoencoder (AE) with a
  NeRF, in which latent features (instead of colours) are rendered and then
  convolutionally decoded. The resulting latent-space NeRF can produce novel
  views with higher quality than standard colour-space NeRFs, as the AE can
  correct certain visual artifacts, while rendering over three times faster. Our
  work is orthogonal to other techniques for improving NeRF efficiency. Further,
  we can control the tradeoff between efficiency and image quality by shrinking
  the AE architecture, achieving over 13 times faster rendering with only a small
  drop in performance. We hope that our approach can form the basis of an
  efficient, yet high-fidelity, 3D scene representation for downstream tasks,
  especially when retaining differentiability is useful, as in many robotics
  scenarios requiring continual learning.
paper_authors._id: 653f2269eefdcdc9fdb301d9
paper_authors.name: Tristan Aumentado-Armstrong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-30T14:05:26.567Z
paper_authors._id: 637eca3e1618112bdd7f6807
paper_authors.avatarUrl: /avatars/04a3809542241f5315654153dc7b4375.svg
paper_authors.fullname: Tristan Aumentado-Armstrong
paper_authors.type: user
paper_authors.user: taumarm
paper_authors._id: 653f2269eefdcdc9fdb301da
paper_authors.name: Ashkan Mirzaei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-30T14:05:38.174Z
paper_authors._id: 634db15dd00bb5d92c3bd94f
paper_authors.avatarUrl: /avatars/9a6a1231bc5205911272d83527593f1a.svg
paper_authors.fullname: Ashkan Mirzaei
paper_authors.type: user
paper_authors.user: ashmrz
paper_authors._id: 653f2269eefdcdc9fdb301db
paper_authors.name: Marcus A. Brubaker
paper_authors._id: 653f2269eefdcdc9fdb301dc
paper_authors.name: Jonathan Kelly
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-30T14:05:53.563Z
paper_authors._id: 64499189d2a85ea9466ab803
paper_authors.avatarUrl: /avatars/fc90bf3a1a5c896afdfefc3d1932d33a.svg
paper_authors.fullname: Jonathan Kelly
paper_authors.type: user
paper_authors.user: Thatguyjk723
paper_authors._id: 653f2269eefdcdc9fdb301dd
paper_authors.name: Alex Levinshtein
paper_authors._id: 653f2269eefdcdc9fdb301de
paper_authors.name: Konstantinos G. Derpanis
paper_authors._id: 653f2269eefdcdc9fdb301df
paper_authors.name: Igor Gilitschenski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-30T14:06:17.824Z
paper_authors._id: 64893f0b9ab28735f1a1ec0b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64893f0b9ab28735f1a1ec0b/2lPBjV16l2qLEWOKU2zwR.jpeg
paper_authors.fullname: Igor Gilitschenski
paper_authors.type: user
paper_authors.user: igilitschenski
paper_title: Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D
    Scene Representations
paper_summary: Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations,
  capable of high quality novel view synthesis of complex scenes. While NeRFs
  have been applied to graphics, vision, and robotics, problems with slow
  rendering speed and characteristic visual artifacts prevent adoption in many
  use cases. In this work, we investigate combining an autoencoder (AE) with a
  NeRF, in which latent features (instead of colours) are rendered and then
  convolutionally decoded. The resulting latent-space NeRF can produce novel
  views with higher quality than standard colour-space NeRFs, as the AE can
  correct certain visual artifacts, while rendering over three times faster. Our
  work is orthogonal to other techniques for improving NeRF efficiency. Further,
  we can control the tradeoff between efficiency and image quality by shrinking
  the AE architecture, achieving over 13 times faster rendering with only a small
  drop in performance. We hope that our approach can form the basis of an
  efficient, yet high-fidelity, 3D scene representation for downstream tasks,
  especially when retaining differentiability is useful, as in many robotics
  scenarios requiring continual learning.
paper_ai_summary: Combining autoencoders with Neural Radiance Fields improves rendering speed and image quality, enabling efficient and high-fidelity 3D scene representation suitable for robotics.
paper_ai_keywords: Neural Radiance Fields NeRFs autoencoder AE latent features convolutional decoding latent-space NeRF visual artifacts novel view synthesis differentiability continual learning