title: SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View
    Consistency
summary: We present Stable Video 4D (SV4D), a latent video diffusion model for
  multi-frame and multi-view consistent dynamic 3D content generation. Unlike
  previous methods that rely on separately trained generative models for video
  generation and novel view synthesis, we design a unified diffusion model to
  generate novel view videos of dynamic 3D objects. Specifically, given a
  monocular reference video, SV4D generates novel views for each video frame that
  are temporally consistent. We then use the generated novel view videos to
  optimize an implicit 4D representation (dynamic NeRF) efficiently, without the
  need for cumbersome SDS-based optimization used in most prior works. To train
  our unified novel view video generation model, we curated a dynamic 3D object
  dataset from the existing Objaverse dataset. Extensive experimental results on
  multiple datasets and user studies demonstrate SV4D's state-of-the-art
  performance on novel-view video synthesis as well as 4D generation compared to
  prior works.
paper_authors._id: 66a1af9f18ce1e6e43377dd1
paper_authors.name: Yiming Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-25T12:23:48.432Z
paper_authors._id: 65c3e6dbe61e74041888f0da
paper_authors.avatarUrl: /avatars/084f51bfc9ead7ebd702e5b53d066f32.svg
paper_authors.fullname: Yiming Xie
paper_authors.type: user
paper_authors.user: ymxie
paper_authors._id: 66a1af9f18ce1e6e43377dd2
paper_authors.name: Chun-Han Yao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-25T12:24:07.251Z
paper_authors._id: 659c2d500868046a55e8fa56
paper_authors.avatarUrl: /avatars/b90d8af7addc0bb3360a22fd14ebfac1.svg
paper_authors.fullname: Chun-Han Yao
paper_authors.type: user
paper_authors.user: chunhanyao
paper_authors._id: 66a1af9f18ce1e6e43377dd3
paper_authors.name: Vikram Voleti
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-25T12:24:12.672Z
paper_authors._id: 631b8e8c8de8e645af6ec059
paper_authors.avatarUrl: /avatars/75eec5fe0e8c18d7974517bf2c8bb511.svg
paper_authors.fullname: Vikram Voleti
paper_authors.type: user
paper_authors.user: vikky2904
paper_authors._id: 66a1af9f18ce1e6e43377dd4
paper_authors.name: Huaizu Jiang
paper_authors._id: 66a1af9f18ce1e6e43377dd5
paper_authors.name: Varun Jampani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-25T12:24:33.345Z
paper_authors._id: 630e7cfe3fc17ffc50f6e835
paper_authors.avatarUrl: /avatars/0b742ff094a09f9374fafcd97ab9e002.svg
paper_authors.fullname: Varun Jampani
paper_authors.type: user
paper_authors.user: varunjampani
paper_title: SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View
    Consistency
paper_summary: We present Stable Video 4D (SV4D), a latent video diffusion model for
  multi-frame and multi-view consistent dynamic 3D content generation. Unlike
  previous methods that rely on separately trained generative models for video
  generation and novel view synthesis, we design a unified diffusion model to
  generate novel view videos of dynamic 3D objects. Specifically, given a
  monocular reference video, SV4D generates novel views for each video frame that
  are temporally consistent. We then use the generated novel view videos to
  optimize an implicit 4D representation (dynamic NeRF) efficiently, without the
  need for cumbersome SDS-based optimization used in most prior works. To train
  our unified novel view video generation model, we curated a dynamic 3D object
  dataset from the existing Objaverse dataset. Extensive experimental results on
  multiple datasets and user studies demonstrate SV4D's state-of-the-art
  performance on novel-view video synthesis as well as 4D generation compared to
  prior works.
paper_ai_summary: Stable Video 4D generates temporally consistent novel views for dynamic 3D objects using a unified latent video diffusion model, outperforming prior methods in both novel-view video synthesis and 4D generation.
paper_ai_keywords: latent video diffusion model multi-frame multi-view dynamic 3D content generation generative models video generation novel view synthesis temporally consistent implicit 4D representation dynamic NeRF SDS-based optimization Objaverse dataset