SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian
  Splatting
We propose SelfSplat, a novel 3D Gaussian Splatting model designed to perform
pose-free and 3D prior-free generalizable 3D reconstruction from unposed
multi-view images. These settings are inherently ill-posed due to the lack of
ground-truth data, learned geometric information, and the need to achieve
accurate 3D reconstruction without finetuning, making it difficult for
conventional methods to achieve high-quality results. Our model addresses these
challenges by effectively integrating explicit 3D representations with
self-supervised depth and pose estimation techniques, resulting in reciprocal
improvements in both pose accuracy and 3D reconstruction quality. Furthermore,
we incorporate a matching-aware pose estimation network and a depth refinement
module to enhance geometry consistency across views, ensuring more accurate and
stable 3D reconstructions. To present the performance of our method, we
evaluated it on large-scale real-world datasets, including RealEstate10K, ACID,
and DL3DV. SelfSplat achieves superior results over previous state-of-the-art
methods in both appearance and geometry quality, also demonstrates strong
cross-dataset generalization capabilities. Extensive ablation studies and
analysis also validate the effectiveness of our proposed methods. Code and
pretrained models are available at https://gynjn.github.io/selfsplat/
67493353a82ab7120ff46295
Gyeongjin Kang
claimed_verified
2024-11-29T09:33:26.851Z
64aa93e799639cc312795ea8
/avatars/904453c39c09aa51d3f6aa9fafb4a47d.svg
Gyeongjin Kang
user
Gynjn
67493353a82ab7120ff46296
Jisang Yoo
claimed_verified
2024-12-12T08:22:46.587Z
675a7d9e06ae644140a57788
/avatars/765ac0e5b443200bd3618399f31a7116.svg
Jisang Yoo
user
jisang1528
67493353a82ab7120ff46297
Jihyeon Park
67493353a82ab7120ff46298
Seungtae Nam
67493353a82ab7120ff46299
Hyeonsoo Im
67493353a82ab7120ff4629a
Sangheon Shin
67493353a82ab7120ff4629b
Sangpil Kim
67493353a82ab7120ff4629c
Eunbyung Park
SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian
  Splatting
We propose SelfSplat, a novel 3D Gaussian Splatting model designed to perform
pose-free and 3D prior-free generalizable 3D reconstruction from unposed
multi-view images. These settings are inherently ill-posed due to the lack of
ground-truth data, learned geometric information, and the need to achieve
accurate 3D reconstruction without finetuning, making it difficult for
conventional methods to achieve high-quality results. Our model addresses these
challenges by effectively integrating explicit 3D representations with
self-supervised depth and pose estimation techniques, resulting in reciprocal
improvements in both pose accuracy and 3D reconstruction quality. Furthermore,
we incorporate a matching-aware pose estimation network and a depth refinement
module to enhance geometry consistency across views, ensuring more accurate and
stable 3D reconstructions. To present the performance of our method, we
evaluated it on large-scale real-world datasets, including RealEstate10K, ACID,
and DL3DV. SelfSplat achieves superior results over previous state-of-the-art
methods in both appearance and geometry quality, also demonstrates strong
cross-dataset generalization capabilities. Extensive ablation studies and
analysis also validate the effectiveness of our proposed methods. Code and
pretrained models are available at https://gynjn.github.io/selfsplat/
SelfSplat integrates 3D Gaussian Splatting with self-supervised depth and pose estimation to achieve high-quality, pose-free, 3D reconstruction from unposed multi-view images, demonstrating superior performance and generalization.
3D Gaussian Splatting self-supervised depth estimation self-supervised pose estimation 3D reconstruction pose-free 3D prior-free geometry consistency appearance quality large-scale real-world datasets RealEstate10K ACID DL3DV ablation studies