SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction
Digitizing 3D static scenes and 4D dynamic events from multi-view images has
long been a challenge in computer vision and graphics. Recently, 3D Gaussian
Splatting (3DGS) has emerged as a practical and scalable reconstruction method,
gaining popularity due to its impressive reconstruction quality, real-time
rendering capabilities, and compatibility with widely used visualization tools.
However, the method requires a substantial number of input views to achieve
high-quality scene reconstruction, introducing a significant practical
bottleneck. This challenge is especially severe in capturing dynamic scenes,
where deploying an extensive camera array can be prohibitively costly. In this
work, we identify the lack of spatial autocorrelation of splat features as one
of the factors contributing to the suboptimal performance of the 3DGS technique
in sparse reconstruction settings. To address the issue, we propose an
optimization strategy that effectively regularizes splat features by modeling
them as the outputs of a corresponding implicit neural field. This results in a
consistent enhancement of reconstruction quality across various scenarios. Our
approach effectively handles static and dynamic cases, as demonstrated by
extensive testing across different setups and scene complexities.
66ea394a6d83fd1e4fcd4483
Marko Mihajlovic
admin_assigned
2024-09-18T08:45:32.293Z
62f7eddeb9fda55613cb9f03
https://cdn-avatars.huggingface.co/v1/production/uploads/1660415445288-noauth.jpeg
Marko Mihajlovic
user
markomih
66ea394a6d83fd1e4fcd4484
Sergey Prokudin
admin_assigned
2024-09-18T08:45:37.796Z
6340364b733f9eef469165ef
/avatars/c86d79f1588bb625b97dcbe471e46def.svg
Sergey Prokudin
user
sprokudin
66ea394a6d83fd1e4fcd4485
Siyu Tang
66ea394a6d83fd1e4fcd4486
Robert Maier
claimed_verified
2024-09-19T07:07:29.707Z
66eb040ee326f844d6442d56
https://cdn-avatars.huggingface.co/v1/production/uploads/66eb040ee326f844d6442d56/o4Rl6tr0uiM0_hqKtkV1X.jpeg
Robert Maier
user
rmaier
66ea394a6d83fd1e4fcd4487
Federica Bogo
66ea394a6d83fd1e4fcd4488
Tony Tung
admin_assigned
2024-09-18T08:46:25.576Z
640c8783536d9fe0f001f202
/avatars/699b9e9d3dd4ab3612b10bf2d00cebeb.svg
Tony Tung
user
Aeriphi
66ea394a6d83fd1e4fcd4489
Edmond Boyer
SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction
Digitizing 3D static scenes and 4D dynamic events from multi-view images has
long been a challenge in computer vision and graphics. Recently, 3D Gaussian
Splatting (3DGS) has emerged as a practical and scalable reconstruction method,
gaining popularity due to its impressive reconstruction quality, real-time
rendering capabilities, and compatibility with widely used visualization tools.
However, the method requires a substantial number of input views to achieve
high-quality scene reconstruction, introducing a significant practical
bottleneck. This challenge is especially severe in capturing dynamic scenes,
where deploying an extensive camera array can be prohibitively costly. In this
work, we identify the lack of spatial autocorrelation of splat features as one
of the factors contributing to the suboptimal performance of the 3DGS technique
in sparse reconstruction settings. To address the issue, we propose an
optimization strategy that effectively regularizes splat features by modeling
them as the outputs of a corresponding implicit neural field. This results in a
consistent enhancement of reconstruction quality across various scenarios. Our
approach effectively handles static and dynamic cases, as demonstrated by
extensive testing across different setups and scene complexities.
The proposed optimization strategy enhances 3D Gaussian Splatting performance in sparse reconstruction settings by modeling splat features using implicit neural fields.
3D Gaussian Splatting 3DGS implicit neural field spatial autocorrelation reconstruction quality real-time rendering visualization tools sparsity dynamic scenes camera array optimization strategy