title: InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding
    and Generation
summary: This paper introduces InternVid, a large-scale video-centric multimodal
  dataset that enables learning powerful and transferable video-text
  representations for multimodal understanding and generation. The InternVid
  dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M
  video clips accompanied by detailed descriptions of total 4.1B words. Our core
  contribution is to develop a scalable approach to autonomously build a
  high-quality video-text dataset with large language models (LLM), thereby
  showcasing its efficacy in learning video-language representation at scale.
  Specifically, we utilize a multi-scale approach to generate video-related
  descriptions. Furthermore, we introduce ViCLIP, a video-text representation
  learning model based on ViT-L. Learned on InternVid via contrastive learning,
  this model demonstrates leading zero-shot action recognition and competitive
  video retrieval performance. Beyond basic video understanding tasks like
  recognition and retrieval, our dataset and model have broad applications. They
  are particularly beneficial for generating interleaved video-text data for
  learning a video-centric dialogue system, advancing video-to-text and
  text-to-video generation research. These proposed resources provide a tool for
  researchers and practitioners interested in multimodal video understanding and
  generation.
paper_authors._id: 64b0a36ad0a5baed1da39714
paper_authors.name: Yi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-18T06:44:41.877Z
paper_authors._id: 63119a987680dc699b2031df
paper_authors.avatarUrl: /avatars/db63ab73806ee9cb8aa01be82d9effdd.svg
paper_authors.fullname: Yi Wang
paper_authors.type: user
paper_authors.user: shepnerd
paper_authors._id: 64b0a36ad0a5baed1da39715
paper_authors.name: Yinan He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-17T06:48:05.499Z
paper_authors._id: 62aafa49f29ff279b51f0182
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62aafa49f29ff279b51f0182/rQx8QFQGOY2qIhqJ8zSRj.jpeg
paper_authors.fullname: yinanhe
paper_authors.type: user
paper_authors.user: ynhe
paper_authors._id: 64b0a36ad0a5baed1da39716
paper_authors.name: Yizhuo Li
paper_authors._id: 64b0a36ad0a5baed1da39717
paper_authors.name: Kunchang Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T10:15:57.763Z
paper_authors._id: 61fb81006374891646732f37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1643872995181-61fb81006374891646732f37.jpeg
paper_authors.fullname: Kunchang Li
paper_authors.type: user
paper_authors.user: Andy1621
paper_authors._id: 64b0a36ad0a5baed1da39718
paper_authors.name: Jiashuo Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-19T11:30:28.806Z
paper_authors._id: 64b89a14cf14c2fabe96664c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/tEd3fBjMcEubF4plqzcUz.jpeg
paper_authors.fullname: Jiashuo Yu
paper_authors.type: user
paper_authors.user: awojustin
paper_authors._id: 64b0a36ad0a5baed1da39719
paper_authors.name: Xin Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-04T10:17:47.388Z
paper_authors._id: 64a51faf9ec0167a99a4d114
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64a51faf9ec0167a99a4d114/IlJS0mwkF_ZIebaQrbJIl.jpeg
paper_authors.fullname: Xin Ma
paper_authors.type: user
paper_authors.user: maxin-cn
paper_authors._id: 64b0a36ad0a5baed1da3971a
paper_authors.name: Xinyuan Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T10:19:13.580Z
paper_authors._id: 643e943a70c6a27621eb1c89
paper_authors.avatarUrl: /avatars/73ec521ab5ba84cc7908c52c0acef6ef.svg
paper_authors.fullname: Xinyuan Chen
paper_authors.type: user
paper_authors.user: AriaChen
paper_authors._id: 64b0a36ad0a5baed1da3971b
paper_authors.name: Yaohui Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-18T06:44:38.033Z
paper_authors._id: 63201256c6b20f03c829c4b8
paper_authors.avatarUrl: /avatars/a42092119777d65e60b12eb5ba0e45f1.svg
paper_authors.fullname: Yaohui Wang
paper_authors.type: user
paper_authors.user: YaohuiW
paper_authors._id: 64b0a36ad0a5baed1da3971c
paper_authors.name: Ping Luo
paper_authors._id: 64b0a36ad0a5baed1da3971d
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T10:21:18.107Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_authors._id: 64b0a36ad0a5baed1da3971e
paper_authors.name: Yali Wang
paper_authors._id: 64b0a36ad0a5baed1da3971f
paper_authors.name: Limin Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T10:22:42.928Z
paper_authors._id: 62c77f4352d8ae531f5511f9
paper_authors.avatarUrl: /avatars/50198ccb02ccd286975a4613fbabee28.svg
paper_authors.fullname: Limin Wang
paper_authors.type: user
paper_authors.user: lmwang
paper_authors._id: 64b0a36ad0a5baed1da39720
paper_authors.name: Yu Qiao
paper_title: InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding
    and Generation
paper_summary: This paper introduces InternVid, a large-scale video-centric multimodal
  dataset that enables learning powerful and transferable video-text
  representations for multimodal understanding and generation. The InternVid
  dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M
  video clips accompanied by detailed descriptions of total 4.1B words. Our core
  contribution is to develop a scalable approach to autonomously build a
  high-quality video-text dataset with large language models (LLM), thereby
  showcasing its efficacy in learning video-language representation at scale.
  Specifically, we utilize a multi-scale approach to generate video-related
  descriptions. Furthermore, we introduce ViCLIP, a video-text representation
  learning model based on ViT-L. Learned on InternVid via contrastive learning,
  this model demonstrates leading zero-shot action recognition and competitive
  video retrieval performance. Beyond basic video understanding tasks like
  recognition and retrieval, our dataset and model have broad applications. They
  are particularly beneficial for generating interleaved video-text data for
  learning a video-centric dialogue system, advancing video-to-text and
  text-to-video generation research. These proposed resources provide a tool for
  researchers and practitioners interested in multimodal video understanding and
  generation.
paper_ai_keywords: InternVid multimodal video-centric video-text large-scale dataset representations understanding generation video clips detailed descriptions large language models (LLM) multi-scale contrastive learning ViCLIP ViT-L zero-shot action recognition video retrieval video-centric dialogue system video-to-text text-to-video generation