NeRF Is a Valuable Assistant for 3D Gaussian Splatting
We introduce NeRF-GS, a novel framework that jointly optimizes Neural
Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). This framework
leverages the inherent continuous spatial representation of NeRF to mitigate
several limitations of 3DGS, including sensitivity to Gaussian initialization,
limited spatial awareness, and weak inter-Gaussian correlations, thereby
enhancing its performance. In NeRF-GS, we revisit the design of 3DGS and
progressively align its spatial features with NeRF, enabling both
representations to be optimized within the same scene through shared 3D spatial
information. We further address the formal distinctions between the two
approaches by optimizing residual vectors for both implicit features and
Gaussian positions to enhance the personalized capabilities of 3DGS.
Experimental results on benchmark datasets show that NeRF-GS surpasses existing
methods and achieves state-of-the-art performance. This outcome confirms that
NeRF and 3DGS are complementary rather than competing, offering new insights
into hybrid approaches that combine 3DGS and NeRF for efficient 3D scene
representation.
688c51de8c434640078cc467
Shuangkang Fang
688c51de8c434640078cc468
I-Chao Shen
688c51de8c434640078cc469
Takeo Igarashi
688c51de8c434640078cc46a
Yufeng Wang
688c51de8c434640078cc46b
ZeSheng Wang
688c51de8c434640078cc46c
Yi Yang
688c51de8c434640078cc46d
Wenrui Ding
688c51de8c434640078cc46e
Shuchang Zhou
NeRF Is a Valuable Assistant for 3D Gaussian Splatting
We introduce NeRF-GS, a novel framework that jointly optimizes Neural
Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). This framework
leverages the inherent continuous spatial representation of NeRF to mitigate
several limitations of 3DGS, including sensitivity to Gaussian initialization,
limited spatial awareness, and weak inter-Gaussian correlations, thereby
enhancing its performance. In NeRF-GS, we revisit the design of 3DGS and
progressively align its spatial features with NeRF, enabling both
representations to be optimized within the same scene through shared 3D spatial
information. We further address the formal distinctions between the two
approaches by optimizing residual vectors for both implicit features and
Gaussian positions to enhance the personalized capabilities of 3DGS.
Experimental results on benchmark datasets show that NeRF-GS surpasses existing
methods and achieves state-of-the-art performance. This outcome confirms that
NeRF and 3DGS are complementary rather than competing, offering new insights
into hybrid approaches that combine 3DGS and NeRF for efficient 3D scene
representation.
NeRF-GS combines Neural Radiance Fields and 3D Gaussian Splatting to enhance 3D scene representation and performance through joint optimization and shared spatial information.
Neural Radiance Fields 3D Gaussian Splatting spatial representation Gaussian initialization spatial awareness inter-Gaussian correlations residual vectors implicit features Gaussian positions hybrid approaches