title: Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for
    Speech Generation
summary: Recent advancements in speech generation have been driven by the large-scale
  training datasets. However, current models fall short of capturing the
  spontaneity and variability inherent in real-world human speech, due to their
  reliance on audiobook datasets limited to formal read-aloud speech styles. To
  bridge this gap, we introduce Emilia-Pipe, an open-source preprocessing
  pipeline to extract high-quality training data from valuable yet underexplored
  in-the-wild data that capture spontaneous human speech in real-world contexts.
  By leveraging Emilia-Pipe, we construct Emilia, the first multilingual speech
  generation dataset derived from in-the-wild speech data. This dataset comprises
  over 101k hours of speech across six languages: English, Chinese, German,
  French, Japanese, and Korean. Besides, we expand Emilia to Emilia-Large, a
  dataset exceeding 216k hours, making it the largest open-source speech
  generation dataset available. Extensive experiments demonstrate that Emilia
  significantly outperforms traditional audiobook datasets in generating
  spontaneous and human-like speech, showcasing superior performance in capturing
  diverse speaker timbre and speaking styles of real-world human speech.
  Furthermore, this work underscores the importance of scaling dataset size to
  advance speech generation research and validates the effectiveness of Emilia
  for both multilingual and crosslingual speech generation.
paper_authors._id: 6798a917a8b0d165e39e17f5
paper_authors.name: Haorui He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-28T13:52:52.095Z
paper_authors._id: 61a7569eaf0333e76eb428a8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61a7569eaf0333e76eb428a8/zwseNheR4Hx0DtCmf_v5H.jpeg
paper_authors.fullname: HarryHe11
paper_authors.type: user
paper_authors.user: HarryHe
paper_authors._id: 6798a917a8b0d165e39e17f6
paper_authors.name: Zengqiang Shang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:12:12.565Z
paper_authors._id: 64b77dd308e2452d18ddd279
paper_authors.avatarUrl: /avatars/258f21fa20a3a187050d80c6088a1f50.svg
paper_authors.fullname: shangzengqiang
paper_authors.type: user
paper_authors.user: clatter-1
paper_authors._id: 6798a917a8b0d165e39e17f7
paper_authors.name: Chaoren Wang
paper_authors._id: 6798a917a8b0d165e39e17f8
paper_authors.name: Xuyuan Li
paper_authors._id: 6798a917a8b0d165e39e17f9
paper_authors.name: Yicheng Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:12:57.932Z
paper_authors._id: 66b5f38a080d890d1727a2a4
paper_authors.avatarUrl: /avatars/4d73017ce888437225d994d8ba370e5d.svg
paper_authors.fullname: guyicheng
paper_authors.type: user
paper_authors.user: guyicheng
paper_authors._id: 6798a917a8b0d165e39e17fa
paper_authors.name: Hua Hua
paper_authors._id: 6798a917a8b0d165e39e17fb
paper_authors.name: Liwei Liu
paper_authors._id: 6798a917a8b0d165e39e17fc
paper_authors.name: Chen Yang
paper_authors._id: 6798a917a8b0d165e39e17fd
paper_authors.name: Jiaqi Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:14:30.597Z
paper_authors._id: 6635a711a5243c9638f5e4df
paper_authors.avatarUrl: /avatars/08651622fc1fd5089551b510be8c4530.svg
paper_authors.fullname: Jiaqi Li
paper_authors.type: user
paper_authors.user: jiaqili3
paper_authors._id: 6798a917a8b0d165e39e17fe
paper_authors.name: Peiyang Shi
paper_authors._id: 6798a917a8b0d165e39e17ff
paper_authors.name: Yuancheng Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:14:50.950Z
paper_authors._id: 63072d60cd148dbc5e49f4dd
paper_authors.avatarUrl: /avatars/ffa61038c0ff20848fbcde7c1c34570e.svg
paper_authors.fullname: Yuancheng Wang
paper_authors.type: user
paper_authors.user: Hecheng0625
paper_authors._id: 6798a917a8b0d165e39e1800
paper_authors.name: Kai Chen
paper_authors._id: 6798a917a8b0d165e39e1801
paper_authors.name: Pengyuan Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:14:57.414Z
paper_authors._id: 65fbe8eb030389a29b87446f
paper_authors.avatarUrl: /avatars/6d3ba153c41945e566b7c2c2d6af6da6.svg
paper_authors.fullname: pengyuan zhang
paper_authors.type: user
paper_authors.user: pengyuan2024
paper_authors._id: 6798a917a8b0d165e39e1802
paper_authors.name: Zhizheng Wu
paper_title: Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for
    Speech Generation
paper_summary: Recent advancements in speech generation have been driven by the large-scale
  training datasets. However, current models fall short of capturing the
  spontaneity and variability inherent in real-world human speech, due to their
  reliance on audiobook datasets limited to formal read-aloud speech styles. To
  bridge this gap, we introduce Emilia-Pipe, an open-source preprocessing
  pipeline to extract high-quality training data from valuable yet underexplored
  in-the-wild data that capture spontaneous human speech in real-world contexts.
  By leveraging Emilia-Pipe, we construct Emilia, the first multilingual speech
  generation dataset derived from in-the-wild speech data. This dataset comprises
  over 101k hours of speech across six languages: English, Chinese, German,
  French, Japanese, and Korean. Besides, we expand Emilia to Emilia-Large, a
  dataset exceeding 216k hours, making it the largest open-source speech
  generation dataset available. Extensive experiments demonstrate that Emilia
  significantly outperforms traditional audiobook datasets in generating
  spontaneous and human-like speech, showcasing superior performance in capturing
  diverse speaker timbre and speaking styles of real-world human speech.
  Furthermore, this work underscores the importance of scaling dataset size to
  advance speech generation research and validates the effectiveness of Emilia
  for both multilingual and crosslingual speech generation.
paper_ai_keywords: sequence-to-sequence in-the-wild multilingual speech generation speech generation dataset crosslingual speech generation