title: VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video
    Generation
summary: Text-to-video generative models convert textual prompts into dynamic visual
  content, offering wide-ranging applications in film production, gaming, and
  education. However, their real-world performance often falls short of user
  expectations. One key reason is that these models have not been trained on
  videos related to some topics users want to create. In this paper, we propose
  VideoUFO, the first Video dataset specifically curated to align with Users'
  FOcus in real-world scenarios. Beyond this, our VideoUFO also features: (1)
  minimal (0.29%) overlap with existing video datasets, and (2) videos
  searched exclusively via YouTube's official API under the Creative Commons
  license. These two attributes provide future researchers with greater freedom
  to broaden their training sources. The VideoUFO comprises over 1.09 million
  video clips, each paired with both a brief and a detailed caption
  (description). Specifically, through clustering, we first identify 1,291
  user-focused topics from the million-scale real text-to-video prompt dataset,
  VidProM. Then, we use these topics to retrieve videos from YouTube, split the
  retrieved videos into clips, and generate both brief and detailed captions for
  each clip. After verifying the clips with specified topics, we are left with
  about 1.09 million video clips. Our experiments reveal that (1) current 16
  text-to-video models do not achieve consistent performance across all
  user-focused topics; and (2) a simple model trained on VideoUFO outperforms
  others on worst-performing topics. The dataset is publicly available at
  https://huggingface.co/datasets/WenhaoWang/VideoUFO under the CC BY 4.0
  License.
paper_authors._id: 67c68f7828a037872c5ce5bb
paper_authors.name: Wenhao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T11:14:37.907Z
paper_authors._id: 62b32a4429a410b7f6b06710
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b32a4429a410b7f6b06710/VzgvmnlYZWuifZTkIkCxy.jpeg
paper_authors.fullname: Wenhao Wang
paper_authors.type: user
paper_authors.user: WenhaoWang
paper_authors._id: 67c68f7828a037872c5ce5bc
paper_authors.name: Yi Yang
paper_title: VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video
    Generation
paper_summary: Text-to-video generative models convert textual prompts into dynamic visual
  content, offering wide-ranging applications in film production, gaming, and
  education. However, their real-world performance often falls short of user
  expectations. One key reason is that these models have not been trained on
  videos related to some topics users want to create. In this paper, we propose
  VideoUFO, the first Video dataset specifically curated to align with Users'
  FOcus in real-world scenarios. Beyond this, our VideoUFO also features: (1)
  minimal (0.29%) overlap with existing video datasets, and (2) videos
  searched exclusively via YouTube's official API under the Creative Commons
  license. These two attributes provide future researchers with greater freedom
  to broaden their training sources. The VideoUFO comprises over 1.09 million
  video clips, each paired with both a brief and a detailed caption
  (description). Specifically, through clustering, we first identify 1,291
  user-focused topics from the million-scale real text-to-video prompt dataset,
  VidProM. Then, we use these topics to retrieve videos from YouTube, split the
  retrieved videos into clips, and generate both brief and detailed captions for
  each clip. After verifying the clips with specified topics, we are left with
  about 1.09 million video clips. Our experiments reveal that (1) current 16
  text-to-video models do not achieve consistent performance across all
  user-focused topics; and (2) a simple model trained on VideoUFO outperforms
  others on worst-performing topics. The dataset is publicly available at
  https://huggingface.co/datasets/WenhaoWang/VideoUFO under the CC BY 4.0
  License.
paper_ai_summary: A new dataset, VideoUFO, addresses real-world limitations in text-to-video generative models by focusing on user-centric topics, enhancing training sources and model performance on poorly-covered subjects.
paper_ai_keywords: text-to-video generative models VideoUFO user-focused topics video datasets clustering YouTube's official API Creative Commons license text-to-video prompt dataset VidProM video clips captions model performance