title: ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of
    Video
summary: The Internet's wealth of content, with up to 60% published in English,
  starkly contrasts the global population, where only 18.8% are English speakers,
  and just 5.1% consider it their native language, leading to disparities in
  online information access. Unfortunately, automated processes for dubbing of
  video - replacing the audio track of a video with a translated alternative -
  remains a complex and challenging task due to pipelines, necessitating precise
  timing, facial movement synchronization, and prosody matching. While end-to-end
  dubbing offers a solution, data scarcity continues to impede the progress of
  both end-to-end and pipeline-based methods. In this work, we introduce
  Anim-400K, a comprehensive dataset of over 425K aligned animated video segments
  in Japanese and English supporting various video-related tasks, including
  automated dubbing, simultaneous translation, guided video summarization, and
  genre/theme/style classification. Our dataset is made publicly available for
  research purposes at https://github.com/davidmchan/Anim400K.
paper_authors._id: 659f6fbec0b775ab966257dd
paper_authors.name: Kevin Cai
paper_authors._id: 659f6fbec0b775ab966257de
paper_authors.name: Chonghua Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-11T11:18:43.130Z
paper_authors._id: 650efa2457343388e5773b67
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/E8ndPzmtufhO9tvr4GsPi.jpeg
paper_authors.fullname: Chonghua Liu
paper_authors.type: user
paper_authors.user: mliu0515
paper_authors._id: 659f6fbec0b775ab966257df
paper_authors.name: David M. Chan
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-01-11T06:35:30.606Z
paper_authors._id: 6388f68c43d8b0797a09ff84
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669920369989-noauth.jpeg
paper_authors.fullname: David Chan
paper_authors.type: user
paper_authors.user: davidchan
paper_title: ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of
    Video
paper_summary: The Internet's wealth of content, with up to 60% published in English,
  starkly contrasts the global population, where only 18.8% are English speakers,
  and just 5.1% consider it their native language, leading to disparities in
  online information access. Unfortunately, automated processes for dubbing of
  video - replacing the audio track of a video with a translated alternative -
  remains a complex and challenging task due to pipelines, necessitating precise
  timing, facial movement synchronization, and prosody matching. While end-to-end
  dubbing offers a solution, data scarcity continues to impede the progress of
  both end-to-end and pipeline-based methods. In this work, we introduce
  Anim-400K, a comprehensive dataset of over 425K aligned animated video segments
  in Japanese and English supporting various video-related tasks, including
  automated dubbing, simultaneous translation, guided video summarization, and
  genre/theme/style classification. Our dataset is made publicly available for
  research purposes at https://github.com/davidmchan/Anim400K.
paper_ai_keywords: aligned animated video segments automated dubbing simultaneous translation guided video summarization genre/theme/style classification