title: LVD-2M: A Long-take Video Dataset with Temporally Dense Captions
summary: The efficacy of video generation models heavily depends on the quality of
  their training datasets. Most previous video generation models are trained on
  short video clips, while recently there has been increasing interest in
  training long video generation models directly on longer videos. However, the
  lack of such high-quality long videos impedes the advancement of long video
  generation. To promote research in long video generation, we desire a new
  dataset with four key features essential for training long video generation
  models: (1) long videos covering at least 10 seconds, (2) long-take videos
  without cuts, (3) large motion and diverse contents, and (4) temporally dense
  captions. To achieve this, we introduce a new pipeline for selecting
  high-quality long-take videos and generating temporally dense captions.
  Specifically, we define a set of metrics to quantitatively assess video quality
  including scene cuts, dynamic degrees, and semantic-level quality, enabling us
  to filter high-quality long-take videos from a large amount of source videos.
  Subsequently, we develop a hierarchical video captioning pipeline to annotate
  long videos with temporally-dense captions. With this pipeline, we curate the
  first long-take video dataset, LVD-2M, comprising 2 million long-take videos,
  each covering more than 10 seconds and annotated with temporally dense
  captions. We further validate the effectiveness of LVD-2M by fine-tuning video
  generation models to generate long videos with dynamic motions. We believe our
  work will significantly contribute to future research in long video generation.
paper_authors._id: 670f3501596612949a2dbff0
paper_authors.name: Tianwei Xiong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-16T09:43:36.713Z
paper_authors._id: 668125557b50b433cda2a211
paper_authors.avatarUrl: /avatars/4e7b5c6ec39e5c9ff43ccc94fe037e4a.svg
paper_authors.fullname: Tianwei Xiong
paper_authors.type: user
paper_authors.user: YuuTennYi
paper_authors._id: 670f3501596612949a2dbff1
paper_authors.name: Yuqing Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-17T11:26:48.804Z
paper_authors._id: 63ea23b9dedfeebe54d02bdf
paper_authors.avatarUrl: /avatars/4d9f9a546aa8c63e277161ea700075c4.svg
paper_authors.fullname: Yuqing Wang
paper_authors.type: user
paper_authors.user: Epiphqny
paper_authors._id: 670f3501596612949a2dbff2
paper_authors.name: Daquan Zhou
paper_authors._id: 670f3501596612949a2dbff3
paper_authors.name: Zhijie Lin
paper_authors._id: 670f3501596612949a2dbff4
paper_authors.name: Jiashi Feng
paper_authors._id: 670f3501596612949a2dbff5
paper_authors.name: Xihui Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-16T09:43:38.536Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_title: LVD-2M: A Long-take Video Dataset with Temporally Dense Captions
paper_summary: The efficacy of video generation models heavily depends on the quality of
  their training datasets. Most previous video generation models are trained on
  short video clips, while recently there has been increasing interest in
  training long video generation models directly on longer videos. However, the
  lack of such high-quality long videos impedes the advancement of long video
  generation. To promote research in long video generation, we desire a new
  dataset with four key features essential for training long video generation
  models: (1) long videos covering at least 10 seconds, (2) long-take videos
  without cuts, (3) large motion and diverse contents, and (4) temporally dense
  captions. To achieve this, we introduce a new pipeline for selecting
  high-quality long-take videos and generating temporally dense captions.
  Specifically, we define a set of metrics to quantitatively assess video quality
  including scene cuts, dynamic degrees, and semantic-level quality, enabling us
  to filter high-quality long-take videos from a large amount of source videos.
  Subsequently, we develop a hierarchical video captioning pipeline to annotate
  long videos with temporally-dense captions. With this pipeline, we curate the
  first long-take video dataset, LVD-2M, comprising 2 million long-take videos,
  each covering more than 10 seconds and annotated with temporally dense
  captions. We further validate the effectiveness of LVD-2M by fine-tuning video
  generation models to generate long videos with dynamic motions. We believe our
  work will significantly contribute to future research in long video generation.