title: FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D
    Reconstruction
summary: Existing sparse-view reconstruction models heavily rely on accurate known
  camera poses. However, deriving camera extrinsics and intrinsics from
  sparse-view images presents significant challenges. In this work, we present
  FreeSplatter, a highly scalable, feed-forward reconstruction framework capable
  of generating high-quality 3D Gaussians from uncalibrated sparse-view images
  and recovering their camera parameters in mere seconds. FreeSplatter is built
  upon a streamlined transformer architecture, comprising sequential
  self-attention blocks that facilitate information exchange among multi-view
  image tokens and decode them into pixel-wise 3D Gaussian primitives. The
  predicted Gaussian primitives are situated in a unified reference frame,
  allowing for high-fidelity 3D modeling and instant camera parameter estimation
  using off-the-shelf solvers. To cater to both object-centric and scene-level
  reconstruction, we train two model variants of FreeSplatter on extensive
  datasets. In both scenarios, FreeSplatter outperforms state-of-the-art
  baselines in terms of reconstruction quality and pose estimation accuracy.
  Furthermore, we showcase FreeSplatter's potential in enhancing the productivity
  of downstream applications, such as text/image-to-3D content creation.
paper_authors._id: 675b9f587a3e15dbc1811e95
paper_authors.name: Jiale Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:07:13.644Z
paper_authors._id: 62c695829db11473f08af1cd
paper_authors.avatarUrl: /avatars/cacb54077892a44aef81454dc107df4f.svg
paper_authors.fullname: Jiale Xu
paper_authors.type: user
paper_authors.user: bluestyle97
paper_authors._id: 675b9f587a3e15dbc1811e96
paper_authors.name: Shenghua Gao
paper_authors._id: 675b9f587a3e15dbc1811e97
paper_authors.name: Ying Shan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-16T11:06:35.611Z
paper_authors._id: 63ca3ddc04c979828310bfcb
paper_authors.avatarUrl: /avatars/615e0d8622950b4408b40d550f02a894.svg
paper_authors.fullname: Ying Shan
paper_authors.type: user
paper_authors.user: yshan2u
paper_title: FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D
    Reconstruction
paper_summary: Existing sparse-view reconstruction models heavily rely on accurate known
  camera poses. However, deriving camera extrinsics and intrinsics from
  sparse-view images presents significant challenges. In this work, we present
  FreeSplatter, a highly scalable, feed-forward reconstruction framework capable
  of generating high-quality 3D Gaussians from uncalibrated sparse-view images
  and recovering their camera parameters in mere seconds. FreeSplatter is built
  upon a streamlined transformer architecture, comprising sequential
  self-attention blocks that facilitate information exchange among multi-view
  image tokens and decode them into pixel-wise 3D Gaussian primitives. The
  predicted Gaussian primitives are situated in a unified reference frame,
  allowing for high-fidelity 3D modeling and instant camera parameter estimation
  using off-the-shelf solvers. To cater to both object-centric and scene-level
  reconstruction, we train two model variants of FreeSplatter on extensive
  datasets. In both scenarios, FreeSplatter outperforms state-of-the-art
  baselines in terms of reconstruction quality and pose estimation accuracy.
  Furthermore, we showcase FreeSplatter's potential in enhancing the productivity
  of downstream applications, such as text/image-to-3D content creation.
paper_ai_keywords: FreeSplatter transformer architecture self-attention blocks 3D Gaussians camera parameters pixel-wise 3D Gaussian primitives unified reference frame off-the-shelf solvers object-centric reconstruction scene-level reconstruction state-of-the-art baselines reconstruction quality pose estimation accuracy text/image-to-3D content creation