title: Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
summary: We introduce Seed-TTS, a family of large-scale autoregressive text-to-speech
  (TTS) models capable of generating speech that is virtually indistinguishable
  from human speech. Seed-TTS serves as a foundation model for speech generation
  and excels in speech in-context learning, achieving performance in speaker
  similarity and naturalness that matches ground truth human speech in both
  objective and subjective evaluations. With fine-tuning, we achieve even higher
  subjective scores across these metrics. Seed-TTS offers superior
  controllability over various speech attributes such as emotion and is capable
  of generating highly expressive and diverse speech for speakers in the wild.
  Furthermore, we propose a self-distillation method for speech factorization, as
  well as a reinforcement learning approach to enhance model robustness, speaker
  similarity, and controllability. We additionally present a non-autoregressive
  (NAR) variant of the Seed-TTS model, named Seed-TTS_DiT, which
  utilizes a fully diffusion-based architecture. Unlike previous NAR-based TTS
  systems, Seed-TTS_DiT does not depend on pre-estimated phoneme
  durations and performs speech generation through end-to-end processing. We
  demonstrate that this variant achieves comparable performance to the language
  model-based variant and showcase its effectiveness in speech editing. We
  encourage readers to listen to demos at
  https://bytedancespeech.github.io/seedtts_tech_report.
paper_authors._id: 665fe7555a8aa91644bf73ce
paper_authors.name: Philip Anastassiou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-06T06:56:13.732Z
paper_authors._id: 6660a20bc950dd103f7d0a86
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/DRt2orjnZ2hlweZFkUHwr.jpeg
paper_authors.fullname: Philip Anastassiou
paper_authors.type: user
paper_authors.user: philip-anastassiou
paper_authors._id: 665fe7555a8aa91644bf73cf
paper_authors.name: Jiawei Chen
paper_authors._id: 665fe7555a8aa91644bf73d0
paper_authors.name: Jitong Chen
paper_authors._id: 665fe7555a8aa91644bf73d1
paper_authors.name: Yuanzhe Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T08:37:50.391Z
paper_authors._id: 64478d8a6ffed6ece1fc4b44
paper_authors.avatarUrl: /avatars/2c37555844ac8343681d5958d1f893ac.svg
paper_authors.fullname: yuanzhe chen 
paper_authors.type: user
paper_authors.user: qq547276542
paper_authors._id: 665fe7555a8aa91644bf73d2
paper_authors.name: Zhuo Chen
paper_authors._id: 665fe7555a8aa91644bf73d3
paper_authors.name: Ziyi Chen
paper_authors._id: 665fe7555a8aa91644bf73d4
paper_authors.name: Jian Cong
paper_authors._id: 665fe7555a8aa91644bf73d5
paper_authors.name: Lelai Deng
paper_authors._id: 665fe7555a8aa91644bf73d6
paper_authors.name: Chuang Ding
paper_authors._id: 665fe7555a8aa91644bf73d7
paper_authors.name: Lu Gao
paper_authors._id: 665fe7555a8aa91644bf73d8
paper_authors.name: Mingqing Gong
paper_authors._id: 665fe7555a8aa91644bf73d9
paper_authors.name: Peisong Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T08:42:24.883Z
paper_authors._id: 6623373128327e7218a6f354
paper_authors.avatarUrl: /avatars/e5aeaafabf17f818c82d98b0854c351a.svg
paper_authors.fullname: huangpeisong
paper_authors.type: user
paper_authors.user: huangpeisong
paper_authors._id: 665fe7555a8aa91644bf73da
paper_authors.name: Qingqing Huang
paper_authors._id: 665fe7555a8aa91644bf73db
paper_authors.name: Zhiying Huang
paper_authors._id: 665fe7555a8aa91644bf73dc
paper_authors.name: Yuanyuan Huo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T08:43:21.818Z
paper_authors._id: 658ab5788ea79e4df4e05f8b
paper_authors.avatarUrl: /avatars/fa560b7046ce488d3294e0f094beb775.svg
paper_authors.fullname: Doris
paper_authors.type: user
paper_authors.user: Yuanyuanhuo
paper_authors._id: 665fe7555a8aa91644bf73dd
paper_authors.name: Dongya Jia
paper_authors._id: 665fe7555a8aa91644bf73de
paper_authors.name: Chumin Li
paper_authors._id: 665fe7555a8aa91644bf73df
paper_authors.name: Feiya Li
paper_authors._id: 665fe7555a8aa91644bf73e0
paper_authors.name: Hui Li
paper_authors._id: 665fe7555a8aa91644bf73e1
paper_authors.name: Jiaxin Li
paper_authors._id: 665fe7555a8aa91644bf73e2
paper_authors.name: Xiaoyang Li
paper_authors._id: 665fe7555a8aa91644bf73e3
paper_authors.name: Xingxing Li
paper_authors._id: 665fe7555a8aa91644bf73e4
paper_authors.name: Lin Liu
paper_authors._id: 665fe7555a8aa91644bf73e5
paper_authors.name: Shouda Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T08:48:22.921Z
paper_authors._id: 65db1dd1ab2f64915ceb6bfc
paper_authors.avatarUrl: /avatars/6c9c54ca7de453555df498ff26e85079.svg
paper_authors.fullname: liushouda
paper_authors.type: user
paper_authors.user: vsooda
paper_authors._id: 665fe7555a8aa91644bf73e6
paper_authors.name: Sichao Liu
paper_authors._id: 665fe7555a8aa91644bf73e7
paper_authors.name: Xudong Liu
paper_authors._id: 665fe7555a8aa91644bf73e8
paper_authors.name: Yuchen Liu
paper_authors._id: 665fe7555a8aa91644bf73e9
paper_authors.name: Zhengxi Liu
paper_authors._id: 665fe7555a8aa91644bf73ea
paper_authors.name: Lu Lu
paper_authors._id: 665fe7555a8aa91644bf73eb
paper_authors.name: Junjie Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T09:01:58.289Z
paper_authors._id: 65a3de338ac06028202c4eca
paper_authors.avatarUrl: /avatars/6a879d96acfdb57c4cf9773b063153bd.svg
paper_authors.fullname: PanJunjie
paper_authors.type: user
paper_authors.user: Sunny8611
paper_authors._id: 665fe7555a8aa91644bf73ec
paper_authors.name: Xin Wang
paper_authors._id: 665fe7555a8aa91644bf73ed
paper_authors.name: Yuping Wang
paper_authors._id: 665fe7555a8aa91644bf73ee
paper_authors.name: Yuxuan Wang
paper_authors._id: 665fe7555a8aa91644bf73ef
paper_authors.name: Zhen Wei
paper_authors._id: 665fe7555a8aa91644bf73f0
paper_authors.name: Jian Wu
paper_authors._id: 665fe7555a8aa91644bf73f1
paper_authors.name: Chao Yao
paper_authors._id: 665fe7555a8aa91644bf73f2
paper_authors.name: Yifeng Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T09:14:14.705Z
paper_authors._id: 66541f311b1ce9f4a6c8e7a6
paper_authors.avatarUrl: /avatars/1062a9887325c426863afa05620a81f8.svg
paper_authors.fullname: YIFENG YANG
paper_authors.type: user
paper_authors.user: cnyangyifeng
paper_authors._id: 665fe7555a8aa91644bf73f3
paper_authors.name: Yuanhao Yi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T09:27:39.142Z
paper_authors._id: 63d903109dfcfa941d4f7508
paper_authors.avatarUrl: /avatars/580c1ff862b35d3efc321c2ebd78efa9.svg
paper_authors.fullname: Yuanhao Yi
paper_authors.type: user
paper_authors.user: YuanhaoYi
paper_authors._id: 665fe7555a8aa91644bf73f4
paper_authors.name: Junteng Zhang
paper_authors._id: 665fe7555a8aa91644bf73f5
paper_authors.name: Qidi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-05T09:14:55.783Z
paper_authors._id: 65ee558f7cca5fa91c25e36f
paper_authors.avatarUrl: /avatars/38830ace279ade1b6fec96b65e642b16.svg
paper_authors.fullname: Qidi Zhang
paper_authors.type: user
paper_authors.user: Austinzqd
paper_authors._id: 665fe7555a8aa91644bf73f6
paper_authors.name: Shuo Zhang
paper_authors._id: 665fe7555a8aa91644bf73f7
paper_authors.name: Wenjie Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-06T06:56:16.616Z
paper_authors._id: 6319c668615c77c25d667d78
paper_authors.avatarUrl: /avatars/4f402c3a57f2e0bb13f05f29836bf558.svg
paper_authors.fullname: zwj
paper_authors.type: user
paper_authors.user: chenxizwj
paper_authors._id: 665fe7555a8aa91644bf73f8
paper_authors.name: Yang Zhang
paper_authors._id: 665fe7555a8aa91644bf73f9
paper_authors.name: Zilin Zhao
paper_authors._id: 665fe7555a8aa91644bf73fa
paper_authors.name: Dejian Zhong
paper_authors._id: 665fe7555a8aa91644bf73fb
paper_authors.name: Xiaobin Zhuang
paper_title: Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
paper_summary: We introduce Seed-TTS, a family of large-scale autoregressive text-to-speech
  (TTS) models capable of generating speech that is virtually indistinguishable
  from human speech. Seed-TTS serves as a foundation model for speech generation
  and excels in speech in-context learning, achieving performance in speaker
  similarity and naturalness that matches ground truth human speech in both
  objective and subjective evaluations. With fine-tuning, we achieve even higher
  subjective scores across these metrics. Seed-TTS offers superior
  controllability over various speech attributes such as emotion and is capable
  of generating highly expressive and diverse speech for speakers in the wild.
  Furthermore, we propose a self-distillation method for speech factorization, as
  well as a reinforcement learning approach to enhance model robustness, speaker
  similarity, and controllability. We additionally present a non-autoregressive
  (NAR) variant of the Seed-TTS model, named Seed-TTS_DiT, which
  utilizes a fully diffusion-based architecture. Unlike previous NAR-based TTS
  systems, Seed-TTS_DiT does not depend on pre-estimated phoneme
  durations and performs speech generation through end-to-end processing. We
  demonstrate that this variant achieves comparable performance to the language
  model-based variant and showcase its effectiveness in speech editing. We
  encourage readers to listen to demos at
  https://bytedancespeech.github.io/seedtts_tech_report.
paper_ai_keywords: tuning-personalization autoregressive text-to-speech (TTS) speech generation seed-tts in-context learning speech similarity naturalness fine-tuning self-distillation method speech factorization reinforcement learning model robustness non-autoregressive (NAR) Seed-TTS_DiT fully diffusion-based architecture phoneme durations end-to-end processing speech editing