title: EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video
    Generation
summary: Video generation has emerged as a promising tool for world simulation,
  leveraging visual data to replicate real-world environments. Within this
  context, egocentric video generation, which centers on the human perspective,
  holds significant potential for enhancing applications in virtual reality,
  augmented reality, and gaming. However, the generation of egocentric videos
  presents substantial challenges due to the dynamic nature of egocentric
  viewpoints, the intricate diversity of actions, and the complex variety of
  scenes encountered. Existing datasets are inadequate for addressing these
  challenges effectively. To bridge this gap, we present EgoVid-5M, the first
  high-quality dataset specifically curated for egocentric video generation.
  EgoVid-5M encompasses 5 million egocentric video clips and is enriched with
  detailed action annotations, including fine-grained kinematic control and
  high-level textual descriptions. To ensure the integrity and usability of the
  dataset, we implement a sophisticated data cleaning pipeline designed to
  maintain frame consistency, action coherence, and motion smoothness under
  egocentric conditions. Furthermore, we introduce EgoDreamer, which is capable
  of generating egocentric videos driven simultaneously by action descriptions
  and kinematic control signals. The EgoVid-5M dataset, associated action
  annotations, and all data cleansing metadata will be released for the
  advancement of research in egocentric video generation.
paper_authors._id: 67357e302a3c30eac4c75625
paper_authors.name: Xiaofeng Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:21:57.290Z
paper_authors._id: 6426616ea5ec4a5cbc535634
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6426616ea5ec4a5cbc535634/hH6JsxnXeakH3mBTeNNmO.jpeg
paper_authors.fullname: JeffWang
paper_authors.type: user
paper_authors.user: Jeff-Wang
paper_authors._id: 67357e302a3c30eac4c75626
paper_authors.name: Kang Zhao
paper_authors._id: 67357e302a3c30eac4c75627
paper_authors.name: Feng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-02T09:33:47.443Z
paper_authors._id: 661658b9e398e4f0757221ce
paper_authors.avatarUrl: /avatars/9e9d55145661d00e2a9e029ba2e085d8.svg
paper_authors.fullname: Liu Feng
paper_authors.type: user
paper_authors.user: LiewFeng
paper_authors._id: 67357e302a3c30eac4c75628
paper_authors.name: Jiayu Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:15:26.293Z
paper_authors._id: 64ec8ce1a91f5a573e4ab454
paper_authors.avatarUrl: /avatars/1c855b26abe040a4d19ade2be29fc1c2.svg
paper_authors.fullname: Jiayu Wang
paper_authors.type: user
paper_authors.user: ArseneRean
paper_authors._id: 67357e302a3c30eac4c75629
paper_authors.name: Guosheng Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:14:35.900Z
paper_authors._id: 654c7f32386fc5525c1ebc12
paper_authors.avatarUrl: /avatars/d2cf2c7c481e6e6f5d15056f1ec0ef10.svg
paper_authors.fullname: zhao guosheng
paper_authors.type: user
paper_authors.user: f1yfish
paper_authors._id: 67357e302a3c30eac4c7562a
paper_authors.name: Xiaoyi Bao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:14:19.541Z
paper_authors._id: 6615fc38d6af2bbe8bd86a72
paper_authors.avatarUrl: /avatars/812277f2f5fd673f2e18988cd499c74d.svg
paper_authors.fullname: Xiaoyi Bao
paper_authors.type: user
paper_authors.user: Shawnee-bxy
paper_authors._id: 67357e302a3c30eac4c7562b
paper_authors.name: Zheng Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:14:10.066Z
paper_authors._id: 656e9b562cd7a3e348011d26
paper_authors.avatarUrl: /avatars/bcca51bdc27c664f8f132420e6ed99fa.svg
paper_authors.fullname: Zheng Zhu
paper_authors.type: user
paper_authors.user: ZhengZhu
paper_authors._id: 67357e302a3c30eac4c7562c
paper_authors.name: Yingya Zhang
paper_authors._id: 67357e302a3c30eac4c7562d
paper_authors.name: Xingang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-15T14:13:53.462Z
paper_authors._id: 64f8118bc3c12b377c976dd0
paper_authors.avatarUrl: /avatars/0bb606bba6d8e91745be66eb9942cad1.svg
paper_authors.fullname: Xingang Wang
paper_authors.type: user
paper_authors.user: purityWang
paper_title: EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video
    Generation
paper_summary: Video generation has emerged as a promising tool for world simulation,
  leveraging visual data to replicate real-world environments. Within this
  context, egocentric video generation, which centers on the human perspective,
  holds significant potential for enhancing applications in virtual reality,
  augmented reality, and gaming. However, the generation of egocentric videos
  presents substantial challenges due to the dynamic nature of egocentric
  viewpoints, the intricate diversity of actions, and the complex variety of
  scenes encountered. Existing datasets are inadequate for addressing these
  challenges effectively. To bridge this gap, we present EgoVid-5M, the first
  high-quality dataset specifically curated for egocentric video generation.
  EgoVid-5M encompasses 5 million egocentric video clips and is enriched with
  detailed action annotations, including fine-grained kinematic control and
  high-level textual descriptions. To ensure the integrity and usability of the
  dataset, we implement a sophisticated data cleaning pipeline designed to
  maintain frame consistency, action coherence, and motion smoothness under
  egocentric conditions. Furthermore, we introduce EgoDreamer, which is capable
  of generating egocentric videos driven simultaneously by action descriptions
  and kinematic control signals. The EgoVid-5M dataset, associated action
  annotations, and all data cleansing metadata will be released for the
  advancement of research in egocentric video generation.
paper_ai_keywords: egocentric video generation virtual reality augmented reality gaming dynamic nature intricate diversity complex variety EgoVid-5M egocentric video clips fine-grained kinematic control high-level textual descriptions data cleaning pipeline frame consistency action coherence motion smoothness egocentric conditions EgoDreamer action descriptions kinematic control signals