title: VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams
summary: Neural Radiance Fields (NeRFs) excel in photorealistically rendering static
  scenes. However, rendering dynamic, long-duration radiance fields on ubiquitous
  devices remains challenging, due to data storage and computational constraints.
  In this paper, we introduce VideoRF, the first approach to enable real-time
  streaming and rendering of dynamic radiance fields on mobile platforms. At the
  core is a serialized 2D feature image stream representing the 4D radiance field
  all in one. We introduce a tailored training scheme directly applied to this 2D
  domain to impose the temporal and spatial redundancy of the feature image
  stream. By leveraging the redundancy, we show that the feature image stream can
  be efficiently compressed by 2D video codecs, which allows us to exploit video
  hardware accelerators to achieve real-time decoding. On the other hand, based
  on the feature image stream, we propose a novel rendering pipeline for VideoRF,
  which has specialized space mappings to query radiance properties efficiently.
  Paired with a deferred shading model, VideoRF has the capability of real-time
  rendering on mobile devices thanks to its efficiency. We have developed a
  real-time interactive player that enables online streaming and rendering of
  dynamic scenes, offering a seamless and immersive free-viewpoint experience
  across a range of devices, from desktops to mobile phones.
paper_authors._id: 656e9da57e3e612b355b0995
paper_authors.name: Liao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:32:07.385Z
paper_authors._id: 64180cb3db24526c7c9b011e
paper_authors.avatarUrl: /avatars/bcbfe1e18a2c6402d5db14520279b04b.svg
paper_authors.fullname: wliao
paper_authors.type: user
paper_authors.user: liaowang11
paper_authors._id: 656e9da57e3e612b355b0996
paper_authors.name: Kaixin Yao
paper_authors._id: 656e9da57e3e612b355b0997
paper_authors.name: Chengcheng Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:32:31.703Z
paper_authors._id: 64e4a9e6b78bc92221b2909e
paper_authors.avatarUrl: /avatars/b57b879d43fafa457cbad77dc4d387b1.svg
paper_authors.fullname: fy
paper_authors.type: user
paper_authors.user: chengchengguo123
paper_authors._id: 656e9da57e3e612b355b0998
paper_authors.name: Zhirui Zhang
paper_authors._id: 656e9da57e3e612b355b0999
paper_authors.name: Qiang Hu
paper_authors._id: 656e9da57e3e612b355b099a
paper_authors.name: Jingyi Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-05T09:33:14.258Z
paper_authors._id: 63ffcec6e7767a89533b8f80
paper_authors.avatarUrl: /avatars/b9e8673943741c2e7b80b15a061bb951.svg
paper_authors.fullname: JINGYI Yu
paper_authors.type: user
paper_authors.user: Daluuu
paper_authors._id: 656e9da57e3e612b355b099b
paper_authors.name: Lan Xu
paper_authors._id: 656e9da57e3e612b355b099c
paper_authors.name: Minye Wu
paper_title: VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams
paper_summary: Neural Radiance Fields (NeRFs) excel in photorealistically rendering static
  scenes. However, rendering dynamic, long-duration radiance fields on ubiquitous
  devices remains challenging, due to data storage and computational constraints.
  In this paper, we introduce VideoRF, the first approach to enable real-time
  streaming and rendering of dynamic radiance fields on mobile platforms. At the
  core is a serialized 2D feature image stream representing the 4D radiance field
  all in one. We introduce a tailored training scheme directly applied to this 2D
  domain to impose the temporal and spatial redundancy of the feature image
  stream. By leveraging the redundancy, we show that the feature image stream can
  be efficiently compressed by 2D video codecs, which allows us to exploit video
  hardware accelerators to achieve real-time decoding. On the other hand, based
  on the feature image stream, we propose a novel rendering pipeline for VideoRF,
  which has specialized space mappings to query radiance properties efficiently.
  Paired with a deferred shading model, VideoRF has the capability of real-time
  rendering on mobile devices thanks to its efficiency. We have developed a
  real-time interactive player that enables online streaming and rendering of
  dynamic scenes, offering a seamless and immersive free-viewpoint experience
  across a range of devices, from desktops to mobile phones.
paper_ai_keywords: Neural Radiance Fields (NeRFs) radiance fields serialized 2D feature image stream temporal and spatial redundancy 2D video codecs video hardware accelerators real-time decoding novel rendering pipeline space mappings deferred shading model real-time rendering real-time interactive player online streaming free-viewpoint experience