RelitLRM: Generative Relightable Radiance for Large Reconstruction Models
<|reference_start|>RelitLRM: Generative Relightable Radiance for Large Reconstruction Models: We propose RelitLRM, a Large Reconstruction Model (LRM) for generating high-quality Gaussian splatting representations of 3D objects under novel illuminations from sparse (4-8) posed images captured under unknown static lighting. Unlike prior inverse rendering methods requiring dense captures and slow optimization, often causing artifacts like incorrect highlights or shadow baking, RelitLRM adopts a feed-forward transformer-based model with a novel combination of a geometry reconstructor and a relightable appearance generator based on diffusion. The model is trained end-to-end on synthetic multi-view renderings of objects under varying known illuminations. This architecture design enables to effectively decompose geometry and appearance, resolve the ambiguity between material and lighting, and capture the multi-modal distribution of shadows and specularity in the relit appearance. We show our sparse-view feed-forward RelitLRM offers competitive relighting results to state-of-the-art dense-view optimization-based baselines while being significantly faster. Our project page is available at: https://relit-lrm.github.io/.<|reference_end|>
@article{zhang2024relitlrm:,
    title={RelitLRM: Generative Relightable Radiance for Large Reconstruction
  Models},
    author={Tianyuan Zhang, Zhengfei Kuang, Haian Jin, Zexiang Xu, Sai Bi, Hao
  Tan, He Zhang, Yiwei Hu, Milos Hasan, William T. Freeman, Kai Zhang, Fujun
  Luan},
    journal={arXiv preprint arXiv:2410.06231},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.06231},
    primaryClass={cs.CV cs.GR cs.LG}
}