title: VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion
    Models
summary: This paper presents a novel paradigm for building scalable 3D generative
  models utilizing pre-trained video diffusion models. The primary obstacle in
  developing foundation 3D generative models is the limited availability of 3D
  data. Unlike images, texts, or videos, 3D data are not readily accessible and
  are difficult to acquire. This results in a significant disparity in scale
  compared to the vast quantities of other types of data. To address this issue,
  we propose using a video diffusion model, trained with extensive volumes of
  text, images, and videos, as a knowledge source for 3D data. By unlocking its
  multi-view generative capabilities through fine-tuning, we generate a
  large-scale synthetic multi-view dataset to train a feed-forward 3D generative
  model. The proposed model, VFusion3D, trained on nearly 3M synthetic multi-view
  data, can generate a 3D asset from a single image in seconds and achieves
  superior performance when compared to current SOTA feed-forward 3D generative
  models, with users preferring our results over 70% of the time.
paper_authors._id: 65f919e89d2f497c66f6ed00
paper_authors.name: Junlin Han
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-03-19T04:51:53.479Z
paper_authors._id: 6541469f6be058da063d329a
paper_authors.avatarUrl: /avatars/08683e07d89562f2fb5319854b21706c.svg
paper_authors.fullname: Junlin Han
paper_authors.type: user
paper_authors.user: JunlinHan
paper_authors._id: 65f919e89d2f497c66f6ed01
paper_authors.name: Filippos Kokkinos
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-03-19T04:51:53.479Z
paper_authors._id: 64918e6bd0aa14078dee0929
paper_authors.avatarUrl: /avatars/407d6996d5d297ad3492a9a2382205cd.svg
paper_authors.fullname: Filippos
paper_authors.type: user
paper_authors.user: fkokkinos
paper_authors._id: 65f919e89d2f497c66f6ed02
paper_authors.name: Philip Torr
paper_title: VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion
    Models
paper_summary: This paper presents a novel paradigm for building scalable 3D generative
  models utilizing pre-trained video diffusion models. The primary obstacle in
  developing foundation 3D generative models is the limited availability of 3D
  data. Unlike images, texts, or videos, 3D data are not readily accessible and
  are difficult to acquire. This results in a significant disparity in scale
  compared to the vast quantities of other types of data. To address this issue,
  we propose using a video diffusion model, trained with extensive volumes of
  text, images, and videos, as a knowledge source for 3D data. By unlocking its
  multi-view generative capabilities through fine-tuning, we generate a
  large-scale synthetic multi-view dataset to train a feed-forward 3D generative
  model. The proposed model, VFusion3D, trained on nearly 3M synthetic multi-view
  data, can generate a 3D asset from a single image in seconds and achieves
  superior performance when compared to current SOTA feed-forward 3D generative
  models, with users preferring our results over 70% of the time.
paper_ai_keywords: diffusion models 3D generative models video diffusion model fine-tuning multi-view generative capabilities feed-forward 3D generative model multi-view dataset 3D asset superior performance SOTA feed-forward 3D generative models