VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in
  One Step
Recovering 3D scenes from sparse views is a challenging task due to its
inherent ill-posed problem. Conventional methods have developed specialized
solutions (e.g., geometry regularization or feed-forward deterministic model)
to mitigate the issue. However, they still suffer from performance degradation
by minimal overlap across input views with insufficient visual information.
Fortunately, recent video generative models show promise in addressing this
challenge as they are capable of generating video clips with plausible 3D
structures. Powered by large pretrained video diffusion models, some pioneering
research start to explore the potential of video generative prior and create 3D
scenes from sparse views. Despite impressive improvements, they are limited by
slow inference time and the lack of 3D constraint, leading to inefficiencies
and reconstruction artifacts that do not align with real-world geometry
structure. In this paper, we propose VideoScene to distill the video diffusion
model to generate 3D scenes in one step, aiming to build an efficient and
effective tool to bridge the gap from video to 3D. Specifically, we design a
3D-aware leap flow distillation strategy to leap over time-consuming redundant
information and train a dynamic denoising policy network to adaptively
determine the optimal leap timestep during inference. Extensive experiments
demonstrate that our VideoScene achieves faster and superior 3D scene
generation results than previous video diffusion models, highlighting its
potential as an efficient tool for future video to 3D applications. Project
Page: https://hanyang-21.github.io/VideoScene
67ee01265839c8a023344aee
Hanyang Wang
claimed_verified
2025-04-03T07:52:06.284Z
65c38f6c137aba2aee524989
/avatars/a93e29f55876df3e65e2532972e057e4.svg
Hanyang Wang
user
hanyang-21
67ee01265839c8a023344aef
Fangfu Liu
admin_assigned
2025-04-03T13:46:26.790Z
6505a02f9310ce8c400edc63
/avatars/bbf781594fc8c812316711aa8e2797aa.svg
Fangfu Liu
user
Liuff23
67ee01265839c8a023344af0
Jiawei Chi
admin_assigned
2025-04-03T13:46:47.647Z
66e14cf2fb009ef598305fe5
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/etohfcD9GKaRPir1HRP-5.jpeg
Jiawei Chi
user
chijw
67ee01265839c8a023344af1
Yueqi Duan
admin_assigned
2025-04-03T13:46:55.497Z
66c8131afafc0fc87ca99650
/avatars/a6eeba2ccf011d5c9964fd38f85bd671.svg
Yueqi Duan
user
duanyueqi
VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in
  One Step
Recovering 3D scenes from sparse views is a challenging task due to its
inherent ill-posed problem. Conventional methods have developed specialized
solutions (e.g., geometry regularization or feed-forward deterministic model)
to mitigate the issue. However, they still suffer from performance degradation
by minimal overlap across input views with insufficient visual information.
Fortunately, recent video generative models show promise in addressing this
challenge as they are capable of generating video clips with plausible 3D
structures. Powered by large pretrained video diffusion models, some pioneering
research start to explore the potential of video generative prior and create 3D
scenes from sparse views. Despite impressive improvements, they are limited by
slow inference time and the lack of 3D constraint, leading to inefficiencies
and reconstruction artifacts that do not align with real-world geometry
structure. In this paper, we propose VideoScene to distill the video diffusion
model to generate 3D scenes in one step, aiming to build an efficient and
effective tool to bridge the gap from video to 3D. Specifically, we design a
3D-aware leap flow distillation strategy to leap over time-consuming redundant
information and train a dynamic denoising policy network to adaptively
determine the optimal leap timestep during inference. Extensive experiments
demonstrate that our VideoScene achieves faster and superior 3D scene
generation results than previous video diffusion models, highlighting its
potential as an efficient tool for future video to 3D applications. Project
Page: https://hanyang-21.github.io/VideoScene
VideoScene distills video diffusion models for efficient one-step 3D scene generation from sparse views, enhancing speed and accuracy compared to previous methods.
video diffusion models 3D scenes sparse views leap flow distillation dynamic denoising policy network 3D-aware video generative models