Novel View Extrapolation with Video Diffusion Priors
The field of novel view synthesis has made significant strides thanks to the
development of radiance field methods. However, most radiance field techniques
are far better at novel view interpolation than novel view extrapolation where
the synthesis novel views are far beyond the observed training views. We design
ViewExtrapolator, a novel view synthesis approach that leverages the generative
priors of Stable Video Diffusion (SVD) for realistic novel view extrapolation.
By redesigning the SVD denoising process, ViewExtrapolator refines the
artifact-prone views rendered by radiance fields, greatly enhancing the clarity
and realism of the synthesized novel views. ViewExtrapolator is a generic novel
view extrapolator that can work with different types of 3D rendering such as
views rendered from point clouds when only a single view or monocular video is
available. Additionally, ViewExtrapolator requires no fine-tuning of SVD,
making it both data-efficient and computation-efficient. Extensive experiments
demonstrate the superiority of ViewExtrapolator in novel view extrapolation.
Project page: https://kunhao-liu.github.io/ViewExtrapolator/.
6740bb4ad3ad4510c18306ed
Kunhao Liu
claimed_verified
2024-11-24T22:06:52.908Z
660b79aa822b119f215c3488
https://cdn-avatars.huggingface.co/v1/production/uploads/660b79aa822b119f215c3488/le_EuzAz6mhW0FQ9J0knV.jpeg
Kunhao Liu
user
KunhaoLiu
6740bb4ad3ad4510c18306ee
Ling Shao
6740bb4ad3ad4510c18306ef
Shijian Lu
Novel View Extrapolation with Video Diffusion Priors
The field of novel view synthesis has made significant strides thanks to the
development of radiance field methods. However, most radiance field techniques
are far better at novel view interpolation than novel view extrapolation where
the synthesis novel views are far beyond the observed training views. We design
ViewExtrapolator, a novel view synthesis approach that leverages the generative
priors of Stable Video Diffusion (SVD) for realistic novel view extrapolation.
By redesigning the SVD denoising process, ViewExtrapolator refines the
artifact-prone views rendered by radiance fields, greatly enhancing the clarity
and realism of the synthesized novel views. ViewExtrapolator is a generic novel
view extrapolator that can work with different types of 3D rendering such as
views rendered from point clouds when only a single view or monocular video is
available. Additionally, ViewExtrapolator requires no fine-tuning of SVD,
making it both data-efficient and computation-efficient. Extensive experiments
demonstrate the superiority of ViewExtrapolator in novel view extrapolation.
Project page: https://kunhao-liu.github.io/ViewExtrapolator/.
ViewExtrapolator uses Stable Video Diffusion to improve realistic novel view extrapolation by refining radiance field-generated views without needing fine-tuning.
radiance field methods novel view interpolation novel view extrapolation ViewExtrapolator Stable Video Diffusion SVD denoising process artifact-prone views 3D rendering point clouds monocular video