title: VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video
    Diffusion Models
summary: The arrival of Sora marks a new era for text-to-video diffusion models,
  bringing significant advancements in video generation and potential
  applications. However, Sora, as well as other text-to-video diffusion models,
  highly relies on the prompts, and there is no publicly available dataset
  featuring a study of text-to-video prompts. In this paper, we introduce
  VidProM, the first large-scale dataset comprising 1.67 million unique
  text-to-video prompts from real users. Additionally, the dataset includes 6.69
  million videos generated by four state-of-the-art diffusion models and some
  related data. We initially demonstrate the curation of this large-scale
  dataset, which is a time-consuming and costly process. Subsequently, we show
  how the proposed VidProM differs from DiffusionDB, a large-scale prompt-gallery
  dataset for image generation. Based on the analysis of these prompts, we
  identify the necessity for a new prompt dataset specifically designed for
  text-to-video generation and gain insights into the preferences of real users
  when creating videos. Our large-scale and diverse dataset also inspires many
  exciting new research areas. For instance, to develop better, more efficient,
  and safer text-to-video diffusion models, we suggest exploring text-to-video
  prompt engineering, efficient video generation, and video copy detection for
  diffusion models. We make the collected dataset VidProM publicly available at
  GitHub and Hugging Face under the CC-BY- NC 4.0 License.
paper_authors._id: 65efc52caa1e46404aca4548
paper_authors.name: Wenhao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-12T12:31:18.969Z
paper_authors._id: 62b32a4429a410b7f6b06710
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b32a4429a410b7f6b06710/VzgvmnlYZWuifZTkIkCxy.jpeg
paper_authors.fullname: Wenhao Wang
paper_authors.type: user
paper_authors.user: WenhaoWang
paper_authors._id: 65efc52caa1e46404aca4549
paper_authors.name: Yi Yang
paper_title: VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video
    Diffusion Models
paper_summary: The arrival of Sora marks a new era for text-to-video diffusion models,
  bringing significant advancements in video generation and potential
  applications. However, Sora, as well as other text-to-video diffusion models,
  highly relies on the prompts, and there is no publicly available dataset
  featuring a study of text-to-video prompts. In this paper, we introduce
  VidProM, the first large-scale dataset comprising 1.67 million unique
  text-to-video prompts from real users. Additionally, the dataset includes 6.69
  million videos generated by four state-of-the-art diffusion models and some
  related data. We initially demonstrate the curation of this large-scale
  dataset, which is a time-consuming and costly process. Subsequently, we show
  how the proposed VidProM differs from DiffusionDB, a large-scale prompt-gallery
  dataset for image generation. Based on the analysis of these prompts, we
  identify the necessity for a new prompt dataset specifically designed for
  text-to-video generation and gain insights into the preferences of real users
  when creating videos. Our large-scale and diverse dataset also inspires many
  exciting new research areas. For instance, to develop better, more efficient,
  and safer text-to-video diffusion models, we suggest exploring text-to-video
  prompt engineering, efficient video generation, and video copy detection for
  diffusion models. We make the collected dataset VidProM publicly available at
  GitHub and Hugging Face under the CC-BY- NC 4.0 License.
paper_ai_summary: A large-scale dataset of text-to-video prompts and generated videos opens new research avenues for improving text-to-video diffusion models.
paper_ai_keywords: TEXT-to-video diffusion models VidProM large-scale dataset text-to-video prompts video generation prompt engineering video copy detection