Voyager: Long-Range and World-Consistent Video Diffusion for Explorable
  3D Scene Generation
Real-world applications like video gaming and virtual reality often demand
the ability to model 3D scenes that users can explore along custom camera
trajectories. While significant progress has been made in generating 3D objects
from text or images, creating long-range, 3D-consistent, explorable 3D scenes
remains a complex and challenging problem. In this work, we present Voyager, a
novel video diffusion framework that generates world-consistent 3D point-cloud
sequences from a single image with user-defined camera path. Unlike existing
approaches, Voyager achieves end-to-end scene generation and reconstruction
with inherent consistency across frames, eliminating the need for 3D
reconstruction pipelines (e.g., structure-from-motion or multi-view stereo).
Our method integrates three key components: 1) World-Consistent Video
Diffusion: A unified architecture that jointly generates aligned RGB and depth
video sequences, conditioned on existing world observation to ensure global
coherence 2) Long-Range World Exploration: An efficient world cache with point
culling and an auto-regressive inference with smooth video sampling for
iterative scene extension with context-aware consistency, and 3) Scalable Data
Engine: A video reconstruction pipeline that automates camera pose estimation
and metric depth prediction for arbitrary videos, enabling large-scale, diverse
training data curation without manual 3D annotations. Collectively, these
designs result in a clear improvement over existing methods in visual quality
and geometric accuracy, with versatile applications.
68413366adeec0116d071af2
Tianyu Huang
claimed_verified
2025-06-05T07:26:41.198Z
63425d394c9a81858b36aeb5
/avatars/511ad6a75bd1c10fc510ef527e7f8e5b.svg
Tianyu Huang
user
tyhuang
68413366adeec0116d071af3
Wangguandong Zheng
68413366adeec0116d071af4
Tengfei Wang
68413366adeec0116d071af5
Yuhao Liu
68413366adeec0116d071af6
Zhenwei Wang
claimed_verified
2025-07-31T08:16:26.086Z
6305e9471e731505a76b9099
https://cdn-avatars.huggingface.co/v1/production/uploads/6305e9471e731505a76b9099/w9csjXIEUzrd0sa1FxIXm.jpeg
Zhenwei WANG (Zavier)
user
ZhenweiWang
68413366adeec0116d071af7
Junta Wu
68413366adeec0116d071af8
Jie Jiang
68413366adeec0116d071af9
Hui Li
68413366adeec0116d071afa
Rynson W. H. Lau
68413366adeec0116d071afb
Wangmeng Zuo
68413366adeec0116d071afc
Chunchao Guo
Voyager: Long-Range and World-Consistent Video Diffusion for Explorable
  3D Scene Generation
Real-world applications like video gaming and virtual reality often demand
the ability to model 3D scenes that users can explore along custom camera
trajectories. While significant progress has been made in generating 3D objects
from text or images, creating long-range, 3D-consistent, explorable 3D scenes
remains a complex and challenging problem. In this work, we present Voyager, a
novel video diffusion framework that generates world-consistent 3D point-cloud
sequences from a single image with user-defined camera path. Unlike existing
approaches, Voyager achieves end-to-end scene generation and reconstruction
with inherent consistency across frames, eliminating the need for 3D
reconstruction pipelines (e.g., structure-from-motion or multi-view stereo).
Our method integrates three key components: 1) World-Consistent Video
Diffusion: A unified architecture that jointly generates aligned RGB and depth
video sequences, conditioned on existing world observation to ensure global
coherence 2) Long-Range World Exploration: An efficient world cache with point
culling and an auto-regressive inference with smooth video sampling for
iterative scene extension with context-aware consistency, and 3) Scalable Data
Engine: A video reconstruction pipeline that automates camera pose estimation
and metric depth prediction for arbitrary videos, enabling large-scale, diverse
training data curation without manual 3D annotations. Collectively, these
designs result in a clear improvement over existing methods in visual quality
and geometric accuracy, with versatile applications.
Voyager is a video diffusion framework that generates world-consistent 3D point-cloud sequences from a single image, enabling long-range, consistent 3D scene exploration with user-defined camera paths.
video diffusion world-consistent video diffusion 3D point-cloud sequences camera path end-to-end scene generation consistent frames unified architecture RGB and depth video sequences world observation global coherence long-range world exploration world cache point culling auto-regressive inference smooth video sampling scene extension context-aware consistency scalable data engine camera pose estimation metric depth prediction large-scale diverse training data