title: Wonderland: Navigating 3D Scenes from a Single Image
summary: This paper addresses a challenging question: How can we efficiently create
  high-quality, wide-scope 3D scenes from a single arbitrary image? Existing
  methods face several constraints, such as requiring multi-view data,
  time-consuming per-scene optimization, low visual quality in backgrounds, and
  distorted reconstructions in unseen areas. We propose a novel pipeline to
  overcome these limitations. Specifically, we introduce a large-scale
  reconstruction model that uses latents from a video diffusion model to predict
  3D Gaussian Splattings for the scenes in a feed-forward manner. The video
  diffusion model is designed to create videos precisely following specified
  camera trajectories, allowing it to generate compressed video latents that
  contain multi-view information while maintaining 3D consistency. We train the
  3D reconstruction model to operate on the video latent space with a progressive
  training strategy, enabling the efficient generation of high-quality,
  wide-scope, and generic 3D scenes. Extensive evaluations across various
  datasets demonstrate that our model significantly outperforms existing methods
  for single-view 3D scene generation, particularly with out-of-domain images.
  For the first time, we demonstrate that a 3D reconstruction model can be
  effectively built upon the latent space of a diffusion model to realize
  efficient 3D scene generation.
paper_authors._id: 67610a192504d8a0fbf6a83a
paper_authors.name: Hanwen Liang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-18T09:53:00.045Z
paper_authors._id: 64cb2227e984d09bed90fdc5
paper_authors.avatarUrl: /avatars/15b04cfbd935bd01c84020c8a8e6c8c3.svg
paper_authors.fullname: Hanwen Liang
paper_authors.type: user
paper_authors.user: hw-liang
paper_authors._id: 67610a192504d8a0fbf6a83b
paper_authors.name: Junli Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T11:19:05.096Z
paper_authors._id: 63f54aa73aa49d8cb97b84bc
paper_authors.avatarUrl: /avatars/c73c5870039611ab9162daad46a1ba20.svg
paper_authors.fullname: junli cao
paper_authors.type: user
paper_authors.user: jlcao2
paper_authors._id: 67610a192504d8a0fbf6a83c
paper_authors.name: Vidit Goel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T11:19:00.007Z
paper_authors._id: 636c0c1a15cd58e915bb8139
paper_authors.avatarUrl: /avatars/7c675ac6a7d303d3425e498c4e939eb0.svg
paper_authors.fullname: Vidit Goel
paper_authors.type: user
paper_authors.user: vidit98
paper_authors._id: 67610a192504d8a0fbf6a83d
paper_authors.name: Guocheng Qian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T11:18:51.075Z
paper_authors._id: 645fed74335c21d19f3bf76c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645fed74335c21d19f3bf76c/gwVsllRWtSHbg4a1erkdF.jpeg
paper_authors.fullname: Guocheng Qian
paper_authors.type: user
paper_authors.user: guochengqian
paper_authors._id: 67610a192504d8a0fbf6a83e
paper_authors.name: Sergei Korolev
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T11:18:45.439Z
paper_authors._id: 6338047fe1b78345008dcd22
paper_authors.avatarUrl: /avatars/7306138e83ca642f1638a39efa169c7f.svg
paper_authors.fullname: Sergei Korolev
paper_authors.type: user
paper_authors.user: ZanyRumata
paper_authors._id: 67610a192504d8a0fbf6a83f
paper_authors.name: Demetri Terzopoulos
paper_authors._id: 67610a192504d8a0fbf6a840
paper_authors.name: Konstantinos N. Plataniotis
paper_authors._id: 67610a192504d8a0fbf6a841
paper_authors.name: Sergey Tulyakov
paper_authors._id: 67610a192504d8a0fbf6a842
paper_authors.name: Jian Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T08:49:35.742Z
paper_authors._id: 61f19829233c91cbd2f79e70
paper_authors.avatarUrl: /avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
paper_authors.fullname: Jian Ren
paper_authors.type: user
paper_authors.user: alanspike
paper_title: Wonderland: Navigating 3D Scenes from a Single Image
paper_summary: This paper addresses a challenging question: How can we efficiently create
  high-quality, wide-scope 3D scenes from a single arbitrary image? Existing
  methods face several constraints, such as requiring multi-view data,
  time-consuming per-scene optimization, low visual quality in backgrounds, and
  distorted reconstructions in unseen areas. We propose a novel pipeline to
  overcome these limitations. Specifically, we introduce a large-scale
  reconstruction model that uses latents from a video diffusion model to predict
  3D Gaussian Splattings for the scenes in a feed-forward manner. The video
  diffusion model is designed to create videos precisely following specified
  camera trajectories, allowing it to generate compressed video latents that
  contain multi-view information while maintaining 3D consistency. We train the
  3D reconstruction model to operate on the video latent space with a progressive
  training strategy, enabling the efficient generation of high-quality,
  wide-scope, and generic 3D scenes. Extensive evaluations across various
  datasets demonstrate that our model significantly outperforms existing methods
  for single-view 3D scene generation, particularly with out-of-domain images.
  For the first time, we demonstrate that a 3D reconstruction model can be
  effectively built upon the latent space of a diffusion model to realize
  efficient 3D scene generation.
paper_ai_keywords: large-scale reconstruction model video diffusion model 3D Gaussian Splattings feed-forward manner multi-view information 3D consistency progressive training strategy high-quality 3D scenes wide-scope 3D scenes single-view 3D scene generation out-of-domain images latent space efficient 3D scene generation