title: IllumiNeRF: 3D Relighting without Inverse Rendering
summary: Existing methods for relightable view synthesis -- using a set of images of
  an object under unknown lighting to recover a 3D representation that can be
  rendered from novel viewpoints under a target illumination -- are based on
  inverse rendering, and attempt to disentangle the object geometry, materials,
  and lighting that explain the input images. Furthermore, this typically
  involves optimization through differentiable Monte Carlo rendering, which is
  brittle and computationally-expensive. In this work, we propose a simpler
  approach: we first relight each input image using an image diffusion model
  conditioned on lighting and then reconstruct a Neural Radiance Field (NeRF)
  with these relit images, from which we render novel views under the target
  lighting. We demonstrate that this strategy is surprisingly competitive and
  achieves state-of-the-art results on multiple relighting benchmarks. Please see
  our project page at https://illuminerf.github.io/.
paper_authors._id: 6667cac4576792d8883870cd
paper_authors.name: Xiaoming Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:46:00.557Z
paper_authors._id: 642006ed290342c5df8158cc
paper_authors.avatarUrl: /avatars/408d35ff2512eb93ed6570befa33aa03.svg
paper_authors.fullname: Xiaoming Zhao
paper_authors.type: user
paper_authors.user: xzhao
paper_authors._id: 6667cac4576792d8883870ce
paper_authors.name: Pratul P. Srinivasan
paper_authors._id: 6667cac4576792d8883870cf
paper_authors.name: Dor Verbin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-11T07:46:18.538Z
paper_authors._id: 6315b0fc915d0b806830b424
paper_authors.avatarUrl: /avatars/e25a7c8df2364c6e8024ec5c646a2889.svg
paper_authors.fullname: Dor Verbin
paper_authors.type: user
paper_authors.user: dorverbin
paper_authors._id: 6667cac4576792d8883870d0
paper_authors.name: Keunhong Park
paper_authors._id: 6667cac4576792d8883870d1
paper_authors.name: Ricardo Martin Brualla
paper_authors._id: 6667cac4576792d8883870d2
paper_authors.name: Philipp Henzler
paper_title: IllumiNeRF: 3D Relighting without Inverse Rendering
paper_summary: Existing methods for relightable view synthesis -- using a set of images of
  an object under unknown lighting to recover a 3D representation that can be
  rendered from novel viewpoints under a target illumination -- are based on
  inverse rendering, and attempt to disentangle the object geometry, materials,
  and lighting that explain the input images. Furthermore, this typically
  involves optimization through differentiable Monte Carlo rendering, which is
  brittle and computationally-expensive. In this work, we propose a simpler
  approach: we first relight each input image using an image diffusion model
  conditioned on lighting and then reconstruct a Neural Radiance Field (NeRF)
  with these relit images, from which we render novel views under the target
  lighting. We demonstrate that this strategy is surprisingly competitive and
  achieves state-of-the-art results on multiple relighting benchmarks. Please see
  our project page at https://illuminerf.github.io/.
paper_ai_keywords: relightable view synthesis inverse rendering differentiable Monte Carlo rendering image diffusion model Neural Radiance Field (NeRF) relit images relighting benchmarks