title: CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
summary: We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular
  video. CAT4D leverages a multi-view video diffusion model trained on a diverse
  combination of datasets to enable novel view synthesis at any specified camera
  poses and timestamps. Combined with a novel sampling approach, this model can
  transform a single monocular video into a multi-view video, enabling robust 4D
  reconstruction via optimization of a deformable 3D Gaussian representation. We
  demonstrate competitive performance on novel view synthesis and dynamic scene
  reconstruction benchmarks, and highlight the creative capabilities for 4D scene
  generation from real or generated videos. See our project page for results and
  interactive demos: cat-4d.github.io.
paper_authors._id: 6747e4ce2e5e8d6c8522f6f7
paper_authors.name: Rundi Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-28T09:19:38.809Z
paper_authors._id: 63823d59a18d1ca09653aa21
paper_authors.avatarUrl: /avatars/151e1ca46627abdd99c27945ea2fa441.svg
paper_authors.fullname: Rundi Wu
paper_authors.type: user
paper_authors.user: rundi
paper_authors._id: 6747e4ce2e5e8d6c8522f6f8
paper_authors.name: Ruiqi Gao
paper_authors._id: 6747e4ce2e5e8d6c8522f6f9
paper_authors.name: Ben Poole
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-28T10:35:38.276Z
paper_authors._id: 63213768d2d45f315183abc3
paper_authors.avatarUrl: /avatars/521f52f430b73e514afb06e3a9d6779d.svg
paper_authors.fullname: Ben Poole
paper_authors.type: user
paper_authors.user: doinkda
paper_authors._id: 6747e4ce2e5e8d6c8522f6fa
paper_authors.name: Alex Trevithick
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-28T10:35:32.163Z
paper_authors._id: 656a76839dcedd16d5d4ed49
paper_authors.avatarUrl: /avatars/81e4a17f9af8c990dc13bf1a7bb4efbd.svg
paper_authors.fullname: Alex Trevithick
paper_authors.type: user
paper_authors.user: atrevithick
paper_authors._id: 6747e4ce2e5e8d6c8522f6fb
paper_authors.name: Changxi Zheng
paper_authors._id: 6747e4ce2e5e8d6c8522f6fc
paper_authors.name: Jonathan T. Barron
paper_authors._id: 6747e4ce2e5e8d6c8522f6fd
paper_authors.name: Aleksander Holynski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-28T10:35:58.104Z
paper_authors._id: 62f6e2792e53c2efd33faa92
paper_authors.avatarUrl: /avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
paper_authors.fullname: Aleksander Holynski
paper_authors.type: user
paper_authors.user: holynski
paper_title: CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
paper_summary: We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular
  video. CAT4D leverages a multi-view video diffusion model trained on a diverse
  combination of datasets to enable novel view synthesis at any specified camera
  poses and timestamps. Combined with a novel sampling approach, this model can
  transform a single monocular video into a multi-view video, enabling robust 4D
  reconstruction via optimization of a deformable 3D Gaussian representation. We
  demonstrate competitive performance on novel view synthesis and dynamic scene
  reconstruction benchmarks, and highlight the creative capabilities for 4D scene
  generation from real or generated videos. See our project page for results and
  interactive demos: cat-4d.github.io.
paper_ai_keywords: multi-view video diffusion model novel view synthesis camera poses timestamps sampling approach optimization deformable 3D Gaussian representation dynamic scene reconstruction competitive performance benchmarks creative capabilities