title: SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image
    using Latent Video Diffusion
summary: We present Stable Video 3D (SV3D) -- a latent video diffusion model for
  high-resolution, image-to-multi-view generation of orbital videos around a 3D
  object. Recent work on 3D generation propose techniques to adapt 2D generative
  models for novel view synthesis (NVS) and 3D optimization. However, these
  methods have several disadvantages due to either limited views or inconsistent
  NVS, thereby affecting the performance of 3D object generation. In this work,
  we propose SV3D that adapts image-to-video diffusion model for novel multi-view
  synthesis and 3D generation, thereby leveraging the generalization and
  multi-view consistency of the video models, while further adding explicit
  camera control for NVS. We also propose improved 3D optimization techniques to
  use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental
  results on multiple datasets with 2D and 3D metrics as well as user study
  demonstrate SV3D's state-of-the-art performance on NVS as well as 3D
  reconstruction compared to prior works.
paper_authors._id: 65f90aa244d5f5e014ad1eee
paper_authors.name: Vikram Voleti
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:39:15.493Z
paper_authors._id: 631b8e8c8de8e645af6ec059
paper_authors.avatarUrl: /avatars/75eec5fe0e8c18d7974517bf2c8bb511.svg
paper_authors.fullname: Vikram Voleti
paper_authors.type: user
paper_authors.user: vikky2904
paper_authors._id: 65f90aa244d5f5e014ad1eef
paper_authors.name: Chun-Han Yao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:39:28.275Z
paper_authors._id: 659c2d500868046a55e8fa56
paper_authors.avatarUrl: /avatars/b90d8af7addc0bb3360a22fd14ebfac1.svg
paper_authors.fullname: Chun-Han Yao
paper_authors.type: user
paper_authors.user: chunhanyao
paper_authors._id: 65f90aa244d5f5e014ad1ef0
paper_authors.name: Mark Boss
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-19T08:45:23.279Z
paper_authors._id: 64b7f06fda8017900e893eb4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b7f06fda8017900e893eb4/3VcQFjERXyAjHQFLMFEJt.jpeg
paper_authors.fullname: Mark Boss
paper_authors.type: user
paper_authors.user: mboss
paper_authors._id: 65f90aa244d5f5e014ad1ef1
paper_authors.name: Adam Letts
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:39:35.094Z
paper_authors._id: 62a43e93d83c3facaf8b81ef
paper_authors.avatarUrl: /avatars/4f710707c790f88bb358992ce863b7b1.svg
paper_authors.fullname: Adam Letts
paper_authors.type: user
paper_authors.user: gandamu
paper_authors._id: 65f90aa244d5f5e014ad1ef2
paper_authors.name: David Pankratz
paper_authors._id: 65f90aa244d5f5e014ad1ef3
paper_authors.name: Dmitry Tochilkin
paper_authors._id: 65f90aa244d5f5e014ad1ef4
paper_authors.name: Christian Laforte
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:39:51.844Z
paper_authors._id: 62f59a712873214eb5f5c2c4
paper_authors.avatarUrl: /avatars/1a518c1ea70d3e60eeac8f4e27b2fcb5.svg
paper_authors.fullname: Christian Laforte
paper_authors.type: user
paper_authors.user: claforte
paper_authors._id: 65f90aa244d5f5e014ad1ef5
paper_authors.name: Robin Rombach
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:39:59.039Z
paper_authors._id: 62cfefa74b3e8dc1e32e38bf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1657794439283-noauth.png
paper_authors.fullname: Robin Rombach
paper_authors.type: user
paper_authors.user: rromb
paper_authors._id: 65f90aa244d5f5e014ad1ef6
paper_authors.name: Varun Jampani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-19T09:40:05.036Z
paper_authors._id: 630e7cfe3fc17ffc50f6e835
paper_authors.avatarUrl: /avatars/0b742ff094a09f9374fafcd97ab9e002.svg
paper_authors.fullname: Varun Jampani
paper_authors.type: user
paper_authors.user: varunjampani
paper_title: SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image
    using Latent Video Diffusion
paper_summary: We present Stable Video 3D (SV3D) -- a latent video diffusion model for
  high-resolution, image-to-multi-view generation of orbital videos around a 3D
  object. Recent work on 3D generation propose techniques to adapt 2D generative
  models for novel view synthesis (NVS) and 3D optimization. However, these
  methods have several disadvantages due to either limited views or inconsistent
  NVS, thereby affecting the performance of 3D object generation. In this work,
  we propose SV3D that adapts image-to-video diffusion model for novel multi-view
  synthesis and 3D generation, thereby leveraging the generalization and
  multi-view consistency of the video models, while further adding explicit
  camera control for NVS. We also propose improved 3D optimization techniques to
  use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental
  results on multiple datasets with 2D and 3D metrics as well as user study
  demonstrate SV3D's state-of-the-art performance on NVS as well as 3D
  reconstruction compared to prior works.
paper_ai_summary: Stable Video 3D (SV3D) achieves high-resolution, multi-view 3D object generation by adapting image-to-video diffusion models and improving 3D optimization techniques.
paper_ai_keywords: latent video diffusion model high-resolution image-to-multi-view generation orbital videos novel view synthesis (NVS) 3D object generation image-to-video diffusion model multi-view consistency camera control image-to-3D generation 3D optimization techniques NVS 3D reconstruction