title: LightSpeed: Light and Fast Neural Light Fields on Mobile Devices
summary: Real-time novel-view image synthesis on mobile devices is prohibitive due to
  the limited computational power and storage. Using volumetric rendering
  methods, such as NeRF and its derivatives, on mobile devices is not suitable
  due to the high computational cost of volumetric rendering. On the other hand,
  recent advances in neural light field representations have shown promising
  real-time view synthesis results on mobile devices. Neural light field methods
  learn a direct mapping from a ray representation to the pixel color. The
  current choice of ray representation is either stratified ray sampling or
  Pl\"{u}cker coordinates, overlooking the classic light slab (two-plane)
  representation, the preferred representation to interpolate between light field
  views. In this work, we find that using the light slab representation is an
  efficient representation for learning a neural light field. More importantly,
  it is a lower-dimensional ray representation enabling us to learn the 4D ray
  space using feature grids which are significantly faster to train and render.
  Although mostly designed for frontal views, we show that the light-slab
  representation can be further extended to non-frontal scenes using a
  divide-and-conquer strategy. Our method offers superior rendering quality
  compared to previous light field methods and achieves a significantly improved
  trade-off between rendering quality and speed.
paper_authors._id: 6539dd390d111541df1f09ad
paper_authors.name: Aarush Gupta
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-26T18:19:16.795Z
paper_authors._id: 653a7dc5d9698d72fd341ae3
paper_authors.avatarUrl: /avatars/51a1f63f25e6d069eb133ceb3b62269e.svg
paper_authors.fullname: Aarush Gupta
paper_authors.type: user
paper_authors.user: aarush98
paper_authors._id: 6539dd390d111541df1f09ae
paper_authors.name: Junli Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T08:29:29.201Z
paper_authors._id: 63f54aa73aa49d8cb97b84bc
paper_authors.avatarUrl: /avatars/c73c5870039611ab9162daad46a1ba20.svg
paper_authors.fullname: junli cao
paper_authors.type: user
paper_authors.user: jlcao2
paper_authors._id: 6539dd390d111541df1f09af
paper_authors.name: Chaoyang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T08:29:36.523Z
paper_authors._id: 64c166de11f3f1d23a99c6fd
paper_authors.avatarUrl: /avatars/d4730f70a6f1cb51266862c7a8d54f77.svg
paper_authors.fullname: Chaoyang Wang
paper_authors.type: user
paper_authors.user: cwang9
paper_authors._id: 6539dd390d111541df1f09b0
paper_authors.name: Ju Hu
paper_authors._id: 6539dd390d111541df1f09b1
paper_authors.name: Sergey Tulyakov
paper_authors._id: 6539dd390d111541df1f09b2
paper_authors.name: Jian Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-26T07:32:31.108Z
paper_authors._id: 61f19829233c91cbd2f79e70
paper_authors.avatarUrl: /avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
paper_authors.fullname: Jian Ren
paper_authors.type: user
paper_authors.user: alanspike
paper_authors._id: 6539dd390d111541df1f09b3
paper_authors.name: László A Jeni
paper_title: LightSpeed: Light and Fast Neural Light Fields on Mobile Devices
paper_summary: Real-time novel-view image synthesis on mobile devices is prohibitive due to
  the limited computational power and storage. Using volumetric rendering
  methods, such as NeRF and its derivatives, on mobile devices is not suitable
  due to the high computational cost of volumetric rendering. On the other hand,
  recent advances in neural light field representations have shown promising
  real-time view synthesis results on mobile devices. Neural light field methods
  learn a direct mapping from a ray representation to the pixel color. The
  current choice of ray representation is either stratified ray sampling or
  Pl\"{u}cker coordinates, overlooking the classic light slab (two-plane)
  representation, the preferred representation to interpolate between light field
  views. In this work, we find that using the light slab representation is an
  efficient representation for learning a neural light field. More importantly,
  it is a lower-dimensional ray representation enabling us to learn the 4D ray
  space using feature grids which are significantly faster to train and render.
  Although mostly designed for frontal views, we show that the light-slab
  representation can be further extended to non-frontal scenes using a
  divide-and-conquer strategy. Our method offers superior rendering quality
  compared to previous light field methods and achieves a significantly improved
  trade-off between rendering quality and speed.
paper_ai_keywords: volumetric rendering NeRF neural light field representations ray representation stratified ray sampling Plücker Coordinates light slab representation neural light field 4D ray space feature grids divide-and-conquer strategy rendering quality enhance-off between rendering quality and speed