title: Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering
    for HDR View Synthesis
summary: Volumetric rendering based methods, like NeRF, excel in HDR view synthesis
  from RAWimages, especially for nighttime scenes. While, they suffer from long
  training times and cannot perform real-time rendering due to dense sampling
  requirements. The advent of 3D Gaussian Splatting (3DGS) enables real-time
  rendering and faster training. However, implementing RAW image-based view
  synthesis directly using 3DGS is challenging due to its inherent drawbacks: 1)
  in nighttime scenes, extremely low SNR leads to poor structure-from-motion
  (SfM) estimation in distant views; 2) the limited representation capacity of
  spherical harmonics (SH) function is unsuitable for RAW linear color space; and
  3) inaccurate scene structure hampers downstream tasks such as refocusing. To
  address these issues, we propose LE3D (Lighting Every darkness with 3DGS). Our
  method proposes Cone Scatter Initialization to enrich the estimation of SfM,
  and replaces SH with a Color MLP to represent the RAW linear color space.
  Additionally, we introduce depth distortion and near-far regularizations to
  improve the accuracy of scene structure for downstream tasks. These designs
  enable LE3D to perform real-time novel view synthesis, HDR rendering,
  refocusing, and tone-mapping changes. Compared to previous volumetric rendering
  based methods, LE3D reduces training time to 1% and improves rendering speed by
  up to 4,000 times for 2K resolution images in terms of FPS. Code and viewer can
  be found in https://github.com/Srameo/LE3D .
paper_authors._id: 6667d23d7dbe2bcda2997580
paper_authors.name: Xin Jin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-11T06:58:58.651Z
paper_authors._id: 6537e8eab01250d1d6efed3a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/gMx73gwdfEhcCFioStGCE.jpeg
paper_authors.fullname: Xin
paper_authors.type: user
paper_authors.user: Srameo
paper_authors._id: 6667d23d7dbe2bcda2997581
paper_authors.name: Pengyi Jiao
paper_authors._id: 6667d23d7dbe2bcda2997582
paper_authors.name: Zheng-Peng Duan
paper_authors._id: 6667d23d7dbe2bcda2997583
paper_authors.name: Xingchao Yang
paper_authors._id: 6667d23d7dbe2bcda2997584
paper_authors.name: Chun-Le Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-11T09:21:14.284Z
paper_authors._id: 66680e227e6e0fee406f57e4
paper_authors.avatarUrl: /avatars/2050193d83949a829afc351d996c737f.svg
paper_authors.fullname: Guo
paper_authors.type: user
paper_authors.user: Chun-Le
paper_authors._id: 6667d23d7dbe2bcda2997585
paper_authors.name: Bo Ren
paper_authors._id: 6667d23d7dbe2bcda2997586
paper_authors.name: Chongyi Li
paper_title: Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering
    for HDR View Synthesis
paper_summary: Volumetric rendering based methods, like NeRF, excel in HDR view synthesis
  from RAWimages, especially for nighttime scenes. While, they suffer from long
  training times and cannot perform real-time rendering due to dense sampling
  requirements. The advent of 3D Gaussian Splatting (3DGS) enables real-time
  rendering and faster training. However, implementing RAW image-based view
  synthesis directly using 3DGS is challenging due to its inherent drawbacks: 1)
  in nighttime scenes, extremely low SNR leads to poor structure-from-motion
  (SfM) estimation in distant views; 2) the limited representation capacity of
  spherical harmonics (SH) function is unsuitable for RAW linear color space; and
  3) inaccurate scene structure hampers downstream tasks such as refocusing. To
  address these issues, we propose LE3D (Lighting Every darkness with 3DGS). Our
  method proposes Cone Scatter Initialization to enrich the estimation of SfM,
  and replaces SH with a Color MLP to represent the RAW linear color space.
  Additionally, we introduce depth distortion and near-far regularizations to
  improve the accuracy of scene structure for downstream tasks. These designs
  enable LE3D to perform real-time novel view synthesis, HDR rendering,
  refocusing, and tone-mapping changes. Compared to previous volumetric rendering
  based methods, LE3D reduces training time to 1% and improves rendering speed by
  up to 4,000 times for 2K resolution images in terms of FPS. Code and viewer can
  be found in https://github.com/Srameo/LE3D .
paper_ai_summary: LE3D enhances 3D Gaussian Splatting for real-time HDR rendering from RAW images, particularly in low light, by improving structure-from-motion and using a Color MLP for color representation.
paper_ai_keywords: NeRF HDR view synthesis RAW images nighttime scenes training times real-time rendering 3D Gaussian Splatting Cone Scatter Initialization spherical harmonics Color MLP depth distortion near-far regularizations novel view synthesis refocusing tone-mapping