title: InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40
    Seconds
summary: While novel view synthesis (NVS) has made substantial progress in 3D computer
  vision, it typically requires an initial estimation of camera intrinsics and
  extrinsics from dense viewpoints. This pre-processing is usually conducted via
  a Structure-from-Motion (SfM) pipeline, a procedure that can be slow and
  unreliable, particularly in sparse-view scenarios with insufficient matched
  features for accurate reconstruction. In this work, we integrate the strengths
  of point-based representations (e.g., 3D Gaussian Splatting, 3D-GS) with
  end-to-end dense stereo models (DUSt3R) to tackle the complex yet unresolved
  issues in NVS under unconstrained settings, which encompasses pose-free and
  sparse view challenges. Our framework, InstantSplat, unifies dense stereo
  priors with 3D-GS to build 3D Gaussians of large-scale scenes from sparseview &
  pose-free images in less than 1 minute. Specifically, InstantSplat comprises a
  Coarse Geometric Initialization (CGI) module that swiftly establishes a
  preliminary scene structure and camera parameters across all training views,
  utilizing globally-aligned 3D point maps derived from a pre-trained dense
  stereo pipeline. This is followed by the Fast 3D-Gaussian Optimization (F-3DGO)
  module, which jointly optimizes the 3D Gaussian attributes and the initialized
  poses with pose regularization. Experiments conducted on the large-scale
  outdoor Tanks & Temples datasets demonstrate that InstantSplat significantly
  improves SSIM (by 32%) while concurrently reducing Absolute Trajectory Error
  (ATE) by 80%. These establish InstantSplat as a viable solution for scenarios
  involving posefree and sparse-view conditions. Project page:
  instantsplat.github.io.
paper_authors._id: 660a2c185294ca0aad3eec0e
paper_authors.name: Zhiwen Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:26:09.118Z
paper_authors._id: 6526386e1c6a09292d8d0a22
paper_authors.avatarUrl: /avatars/471de830de2d775d35368678c1579f87.svg
paper_authors.fullname: fan
paper_authors.type: user
paper_authors.user: Fanzhiwen
paper_authors._id: 660a2c185294ca0aad3eec0f
paper_authors.name: Wenyan Cong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:26:14.831Z
paper_authors._id: 63884e0a3143f4706312f8eb
paper_authors.avatarUrl: /avatars/cd9262a8b5b6e273c629b4d1c76a6d46.svg
paper_authors.fullname: Wenyan Cong
paper_authors.type: user
paper_authors.user: FOGmiaow
paper_authors._id: 660a2c185294ca0aad3eec10
paper_authors.name: Kairun Wen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-18T08:10:49.517Z
paper_authors._id: 646a31bd3eb2bab0419a54ef
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646a31bd3eb2bab0419a54ef/RD2GsZdewSZz7VUwWabUV.png
paper_authors.fullname: Kairun Wen
paper_authors.type: user
paper_authors.user: kairunwen
paper_authors._id: 660a2c185294ca0aad3eec11
paper_authors.name: Kevin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-18T14:58:49.842Z
paper_authors._id: 66b979d9088299999ab714b7
paper_authors.avatarUrl: /avatars/28f941da791e51788a4d8e2321c00ded.svg
paper_authors.fullname: Kevin Wang
paper_authors.type: user
paper_authors.user: Huanyu123
paper_authors._id: 660a2c185294ca0aad3eec12
paper_authors.name: Jian Zhang
paper_authors._id: 660a2c185294ca0aad3eec13
paper_authors.name: Xinghao Ding
paper_authors._id: 660a2c185294ca0aad3eec14
paper_authors.name: Danfei Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:27:25.623Z
paper_authors._id: 636eb10afd9751c3d08279e9
paper_authors.avatarUrl: /avatars/8e9833b2766c76e92be4dba2ceadd61f.svg
paper_authors.fullname: Xu
paper_authors.type: user
paper_authors.user: Danfei
paper_authors._id: 660a2c185294ca0aad3eec15
paper_authors.name: Boris Ivanovic
paper_authors._id: 660a2c185294ca0aad3eec16
paper_authors.name: Marco Pavone
paper_authors._id: 660a2c185294ca0aad3eec17
paper_authors.name: Georgios Pavlakos
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:27:48.501Z
paper_authors._id: 6478d3433b7f8b1f6249b469
paper_authors.avatarUrl: /avatars/11e7d7a94ae26500c1c2ad62e760726f.svg
paper_authors.fullname: Georgios Pavlakos
paper_authors.type: user
paper_authors.user: geopavlakos
paper_authors._id: 660a2c185294ca0aad3eec18
paper_authors.name: Zhangyang Wang
paper_authors._id: 660a2c185294ca0aad3eec19
paper_authors.name: Yue Wang
paper_title: InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40
    Seconds
paper_summary: While novel view synthesis (NVS) has made substantial progress in 3D computer
  vision, it typically requires an initial estimation of camera intrinsics and
  extrinsics from dense viewpoints. This pre-processing is usually conducted via
  a Structure-from-Motion (SfM) pipeline, a procedure that can be slow and
  unreliable, particularly in sparse-view scenarios with insufficient matched
  features for accurate reconstruction. In this work, we integrate the strengths
  of point-based representations (e.g., 3D Gaussian Splatting, 3D-GS) with
  end-to-end dense stereo models (DUSt3R) to tackle the complex yet unresolved
  issues in NVS under unconstrained settings, which encompasses pose-free and
  sparse view challenges. Our framework, InstantSplat, unifies dense stereo
  priors with 3D-GS to build 3D Gaussians of large-scale scenes from sparseview &
  pose-free images in less than 1 minute. Specifically, InstantSplat comprises a
  Coarse Geometric Initialization (CGI) module that swiftly establishes a
  preliminary scene structure and camera parameters across all training views,
  utilizing globally-aligned 3D point maps derived from a pre-trained dense
  stereo pipeline. This is followed by the Fast 3D-Gaussian Optimization (F-3DGO)
  module, which jointly optimizes the 3D Gaussian attributes and the initialized
  poses with pose regularization. Experiments conducted on the large-scale
  outdoor Tanks & Temples datasets demonstrate that InstantSplat significantly
  improves SSIM (by 32%) while concurrently reducing Absolute Trajectory Error
  (ATE) by 80%. These establish InstantSplat as a viable solution for scenarios
  involving posefree and sparse-view conditions. Project page:
  instantsplat.github.io.
paper_ai_keywords: novel view synthesis (NVS) camera intrinsics camera extrinsics Structure-from-Motion (SfM) point-based representations 3D Gaussian Splatting 3D-GS end-to-end dense stereo models DUSt3R InstantSplat Coarse Geometric Initialization (CGI) 3D Gaussians pre-trained dense stereo pipeline globally-aligned 3D point maps Fast 3D-Gaussian Optimization (F-3DGO) pose optimization pose regularization Tanks & Temples datasets SSIM Absolute Trajectory Error (ATE)