FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D
  Reconstruction
Existing sparse-view reconstruction models heavily rely on accurate known
camera poses. However, deriving camera extrinsics and intrinsics from
sparse-view images presents significant challenges. In this work, we present
FreeSplatter, a highly scalable, feed-forward reconstruction framework capable
of generating high-quality 3D Gaussians from uncalibrated sparse-view images
and recovering their camera parameters in mere seconds. FreeSplatter is built
upon a streamlined transformer architecture, comprising sequential
self-attention blocks that facilitate information exchange among multi-view
image tokens and decode them into pixel-wise 3D Gaussian primitives. The
predicted Gaussian primitives are situated in a unified reference frame,
allowing for high-fidelity 3D modeling and instant camera parameter estimation
using off-the-shelf solvers. To cater to both object-centric and scene-level
reconstruction, we train two model variants of FreeSplatter on extensive
datasets. In both scenarios, FreeSplatter outperforms state-of-the-art
baselines in terms of reconstruction quality and pose estimation accuracy.
Furthermore, we showcase FreeSplatter's potential in enhancing the productivity
of downstream applications, such as text/image-to-3D content creation.
675b9f587a3e15dbc1811e95
Jiale Xu
admin_assigned
2024-12-16T11:07:13.644Z
62c695829db11473f08af1cd
/avatars/cacb54077892a44aef81454dc107df4f.svg
Jiale Xu
user
bluestyle97
675b9f587a3e15dbc1811e96
Shenghua Gao
675b9f587a3e15dbc1811e97
Ying Shan
admin_assigned
2024-12-16T11:06:35.611Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D
  Reconstruction
Existing sparse-view reconstruction models heavily rely on accurate known
camera poses. However, deriving camera extrinsics and intrinsics from
sparse-view images presents significant challenges. In this work, we present
FreeSplatter, a highly scalable, feed-forward reconstruction framework capable
of generating high-quality 3D Gaussians from uncalibrated sparse-view images
and recovering their camera parameters in mere seconds. FreeSplatter is built
upon a streamlined transformer architecture, comprising sequential
self-attention blocks that facilitate information exchange among multi-view
image tokens and decode them into pixel-wise 3D Gaussian primitives. The
predicted Gaussian primitives are situated in a unified reference frame,
allowing for high-fidelity 3D modeling and instant camera parameter estimation
using off-the-shelf solvers. To cater to both object-centric and scene-level
reconstruction, we train two model variants of FreeSplatter on extensive
datasets. In both scenarios, FreeSplatter outperforms state-of-the-art
baselines in terms of reconstruction quality and pose estimation accuracy.
Furthermore, we showcase FreeSplatter's potential in enhancing the productivity
of downstream applications, such as text/image-to-3D content creation.
FreeSplatter, a transformer-based feed-forward framework, generates high-quality 3D Gaussians from uncalibrated sparse-view images and quickly estimates camera parameters, outperforming existing models.
streamlined transformer architecture self-attention blocks multi-view image tokens pixel-wise 3D Gaussian primitives unified reference frame off-the-shelf solvers object-centric scene-level reconstruction text/image-to-3D content creation