title: NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to
    Robotics
summary: This work was presented at the IEEE International Conference on Robotics and
  Automation 2023 Workshop on Unconventional Spatial Representations.
    Neural radiance fields (NeRFs) are a class of implicit scene representations
  that model 3D environments from color images. NeRFs are expressive, and can
  model the complex and multi-scale geometry of real world environments, which
  potentially makes them a powerful tool for robotics applications. Modern NeRF
  training libraries can generate a photo-realistic NeRF from a static data set
  in just a few seconds, but are designed for offline use and require a slow pose
  optimization pre-computation step.
    In this work we propose NerfBridge, an open-source bridge between the Robot
  Operating System (ROS) and the popular Nerfstudio library for real-time, online
  training of NeRFs from a stream of images. NerfBridge enables rapid development
  of research on applications of NeRFs in robotics by providing an extensible
  interface to the efficient training pipelines and model libraries provided by
  Nerfstudio. As an example use case we outline a hardware setup that can be used
  NerfBridge to train a NeRF from images captured by a camera mounted to a
  quadrotor in both indoor and outdoor environments.
    For accompanying video https://youtu.be/EH0SLn-RcDg and code
  https://github.com/javieryu/nerf_bridge.
paper_authors._id: 646596410e6c7618f6157389
paper_authors.name: Javier Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-21T20:46:50.752Z
paper_authors._id: 6468d81f188ca2e6bc58dcf8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/aWR-pxpHVa4aJ-5ajk_Iy.jpeg
paper_authors.fullname: Javier Yu
paper_authors.type: user
paper_authors.user: javieryu
paper_authors._id: 646596410e6c7618f615738a
paper_authors.name: Jun En Low
paper_authors._id: 646596410e6c7618f615738b
paper_authors.name: Keiko Nagami
paper_authors._id: 646596410e6c7618f615738c
paper_authors.name: Mac Schwager
paper_title: NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to
    Robotics
paper_summary: This work was presented at the IEEE International Conference on Robotics and
  Automation 2023 Workshop on Unconventional Spatial Representations.
    Neural radiance fields (NeRFs) are a class of implicit scene representations
  that model 3D environments from color images. NeRFs are expressive, and can
  model the complex and multi-scale geometry of real world environments, which
  potentially makes them a powerful tool for robotics applications. Modern NeRF
  training libraries can generate a photo-realistic NeRF from a static data set
  in just a few seconds, but are designed for offline use and require a slow pose
  optimization pre-computation step.
    In this work we propose NerfBridge, an open-source bridge between the Robot
  Operating System (ROS) and the popular Nerfstudio library for real-time, online
  training of NeRFs from a stream of images. NerfBridge enables rapid development
  of research on applications of NeRFs in robotics by providing an extensible
  interface to the efficient training pipelines and model libraries provided by
  Nerfstudio. As an example use case we outline a hardware setup that can be used
  NerfBridge to train a NeRF from images captured by a camera mounted to a
  quadrotor in both indoor and outdoor environments.
    For accompanying video https://youtu.be/EH0SLn-RcDg and code
  https://github.com/javieryu/nerf_bridge.
paper_ai_keywords: Neural radiance fields (NeRFs) implicit scene representations 3D environments color images real-time training online training NeRF training libraries photo-realistic NeRF Robot Operating System (ROS) Nerfstudio library Discriminative redundancy efficient training pipelines model libraries indoor environments outdoor environments