InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40
  Seconds
While novel view synthesis (NVS) has made substantial progress in 3D computer
vision, it typically requires an initial estimation of camera intrinsics and
extrinsics from dense viewpoints. This pre-processing is usually conducted via
a Structure-from-Motion (SfM) pipeline, a procedure that can be slow and
unreliable, particularly in sparse-view scenarios with insufficient matched
features for accurate reconstruction. In this work, we integrate the strengths
of point-based representations (e.g., 3D Gaussian Splatting, 3D-GS) with
end-to-end dense stereo models (DUSt3R) to tackle the complex yet unresolved
issues in NVS under unconstrained settings, which encompasses pose-free and
sparse view challenges. Our framework, InstantSplat, unifies dense stereo
priors with 3D-GS to build 3D Gaussians of large-scale scenes from sparseview &
pose-free images in less than 1 minute. Specifically, InstantSplat comprises a
Coarse Geometric Initialization (CGI) module that swiftly establishes a
preliminary scene structure and camera parameters across all training views,
utilizing globally-aligned 3D point maps derived from a pre-trained dense
stereo pipeline. This is followed by the Fast 3D-Gaussian Optimization (F-3DGO)
module, which jointly optimizes the 3D Gaussian attributes and the initialized
poses with pose regularization. Experiments conducted on the large-scale
outdoor Tanks & Temples datasets demonstrate that InstantSplat significantly
improves SSIM (by 32%) while concurrently reducing Absolute Trajectory Error
(ATE) by 80%. These establish InstantSplat as a viable solution for scenarios
involving posefree and sparse-view conditions. Project page:
instantsplat.github.io.
660a2c185294ca0aad3eec0e
Zhiwen Fan
admin_assigned
2024-08-02T09:26:09.118Z
6526386e1c6a09292d8d0a22
/avatars/471de830de2d775d35368678c1579f87.svg
fan
user
Fanzhiwen
660a2c185294ca0aad3eec0f
Wenyan Cong
admin_assigned
2024-08-02T09:26:14.831Z
63884e0a3143f4706312f8eb
/avatars/cd9262a8b5b6e273c629b4d1c76a6d46.svg
Wenyan Cong
user
FOGmiaow
660a2c185294ca0aad3eec10
Kairun Wen
claimed_verified
2024-04-18T08:10:49.517Z
646a31bd3eb2bab0419a54ef
https://cdn-avatars.huggingface.co/v1/production/uploads/646a31bd3eb2bab0419a54ef/RD2GsZdewSZz7VUwWabUV.png
Kairun Wen
user
kairunwen
660a2c185294ca0aad3eec11
Kevin Wang
claimed_verified
2025-03-18T14:58:49.842Z
66b979d9088299999ab714b7
/avatars/28f941da791e51788a4d8e2321c00ded.svg
Kevin Wang
user
Huanyu123
660a2c185294ca0aad3eec12
Jian Zhang
660a2c185294ca0aad3eec13
Xinghao Ding
660a2c185294ca0aad3eec14
Danfei Xu
admin_assigned
2024-08-02T09:27:25.623Z
636eb10afd9751c3d08279e9
/avatars/8e9833b2766c76e92be4dba2ceadd61f.svg
Xu
user
Danfei
660a2c185294ca0aad3eec15
Boris Ivanovic
660a2c185294ca0aad3eec16
Marco Pavone
660a2c185294ca0aad3eec17
Georgios Pavlakos
admin_assigned
2024-08-02T09:27:48.501Z
6478d3433b7f8b1f6249b469
/avatars/11e7d7a94ae26500c1c2ad62e760726f.svg
Georgios Pavlakos
user
geopavlakos
660a2c185294ca0aad3eec18
Zhangyang Wang
660a2c185294ca0aad3eec19
Yue Wang
InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40
  Seconds
While novel view synthesis (NVS) has made substantial progress in 3D computer
vision, it typically requires an initial estimation of camera intrinsics and
extrinsics from dense viewpoints. This pre-processing is usually conducted via
a Structure-from-Motion (SfM) pipeline, a procedure that can be slow and
unreliable, particularly in sparse-view scenarios with insufficient matched
features for accurate reconstruction. In this work, we integrate the strengths
of point-based representations (e.g., 3D Gaussian Splatting, 3D-GS) with
end-to-end dense stereo models (DUSt3R) to tackle the complex yet unresolved
issues in NVS under unconstrained settings, which encompasses pose-free and
sparse view challenges. Our framework, InstantSplat, unifies dense stereo
priors with 3D-GS to build 3D Gaussians of large-scale scenes from sparseview &
pose-free images in less than 1 minute. Specifically, InstantSplat comprises a
Coarse Geometric Initialization (CGI) module that swiftly establishes a
preliminary scene structure and camera parameters across all training views,
utilizing globally-aligned 3D point maps derived from a pre-trained dense
stereo pipeline. This is followed by the Fast 3D-Gaussian Optimization (F-3DGO)
module, which jointly optimizes the 3D Gaussian attributes and the initialized
poses with pose regularization. Experiments conducted on the large-scale
outdoor Tanks & Temples datasets demonstrate that InstantSplat significantly
improves SSIM (by 32%) while concurrently reducing Absolute Trajectory Error
(ATE) by 80%. These establish InstantSplat as a viable solution for scenarios
involving posefree and sparse-view conditions. Project page:
instantsplat.github.io.
InstantSplat integrates dense stereo models and point-based representations to efficiently reconstruct large-scale scenes from sparse, pose-free images with high SSIM and low ATE.
novel view synthesis SfM pipeline point-based representations 3D Gaussian Splatting DUSt3R dense stereo Coarse Geometric Initialization Fast 3D-Gaussian Optimization Tanks & Temples datasets SSIM Absolute Trajectory Error