title: BASE TTS: Lessons from building a billion-parameter Text-to-Speech model
    on 100K hours of data
summary: We introduce a text-to-speech (TTS) model called BASE TTS, which stands for
  Big Adaptive Streamable TTS with
  Emergent abilities. BASE TTS is the largest TTS model to-date,
  trained on 100K hours of public domain speech data, achieving a new
  state-of-the-art in speech naturalness. It deploys a 1-billion-parameter
  autoregressive Transformer that converts raw texts into discrete codes
  ("speechcodes") followed by a convolution-based decoder which converts these
  speechcodes into waveforms in an incremental, streamable manner. Further, our
  speechcodes are built using a novel speech tokenization technique that features
  speaker ID disentanglement and compression with byte-pair encoding. Echoing the
  widely-reported "emergent abilities" of large language models when trained on
  increasing volume of data, we show that BASE TTS variants built with 10K+ hours
  and 500M+ parameters begin to demonstrate natural prosody on textually complex
  sentences. We design and share a specialized dataset to measure these emergent
  abilities for text-to-speech. We showcase state-of-the-art naturalness of BASE
  TTS by evaluating against baselines that include publicly available large-scale
  text-to-speech systems: YourTTS, Bark and TortoiseTTS. Audio samples generated
  by the model can be heard at https://amazon-ltts-paper.com/.
paper_authors._id: 65cc2d6a11a80579a430d70d
paper_authors.name: Mateusz Łajszczak
paper_authors._id: 65cc2d6a11a80579a430d70e
paper_authors.name: Guillermo Cámbara
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-14T10:11:25.332Z
paper_authors._id: 632862ca27174b0272254f82
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/632862ca27174b0272254f82/TiM-OcoH0jsCQqf0OkWtW.jpeg
paper_authors.fullname: Guillermo Cámbara
paper_authors.type: user
paper_authors.user: gcambara
paper_authors._id: 65cc2d6a11a80579a430d70f
paper_authors.name: Yang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-15T08:28:02.737Z
paper_authors._id: 65cccc8ed4f3ed30d1870e37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65cccc8ed4f3ed30d1870e37/0ETlxoPP187u6XFySa4GZ.jpeg
paper_authors.fullname: Yang Li
paper_authors.type: user
paper_authors.user: lizyacbg
paper_authors._id: 65cc2d6a11a80579a430d710
paper_authors.name: Fatih Beyhan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-14T10:21:13.167Z
paper_authors._id: 5ec973f3968f6028e0559f79
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1628268177397-5ec973f3968f6028e0559f79.jpeg
paper_authors.fullname: Fatih Beyhan
paper_authors.type: user
paper_authors.user: fatihbeyhan
paper_authors._id: 65cc2d6a11a80579a430d711
paper_authors.name: Arent van Korlaar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-15T08:28:00.311Z
paper_authors._id: 64ad769fad6218d51a1305a5
paper_authors.avatarUrl: /avatars/6fd6cede811190a3e725be2c1ed06eaf.svg
paper_authors.fullname: Arent van Korlaar
paper_authors.type: user
paper_authors.user: afvk
paper_authors._id: 65cc2d6a11a80579a430d712
paper_authors.name: Fan Yang
paper_authors._id: 65cc2d6a11a80579a430d713
paper_authors.name: Arnaud Joly
paper_authors._id: 65cc2d6a11a80579a430d714
paper_authors.name: Álvaro Martín-Cortinas
paper_authors._id: 65cc2d6a11a80579a430d715
paper_authors.name: Ammar Abbas
paper_authors._id: 65cc2d6a11a80579a430d716
paper_authors.name: Adam Michalski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-15T08:27:45.676Z
paper_authors._id: 65cd14193f97d5bd5d37f686
paper_authors.avatarUrl: /avatars/cbfe60b0f294c6c6632e5f3d349842f0.svg
paper_authors.fullname: Adam Michalski
paper_authors.type: user
paper_authors.user: micadam
paper_authors._id: 65cc2d6a11a80579a430d717
paper_authors.name: Alexis Moinet
paper_authors._id: 65cc2d6a11a80579a430d718
paper_authors.name: Sri Karlapati
paper_authors._id: 65cc2d6a11a80579a430d719
paper_authors.name: Ewa Muszyńska
paper_authors._id: 65cc2d6a11a80579a430d71a
paper_authors.name: Haohan Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-14T10:25:35.635Z
paper_authors._id: 63783d9d84318944acd305c4
paper_authors.avatarUrl: /avatars/cafd4fd0287e77cc6ebf37c8c8509174.svg
paper_authors.fullname: Haohan Guo
paper_authors.type: user
paper_authors.user: hhguo
paper_authors._id: 65cc2d6a11a80579a430d71b
paper_authors.name: Bartosz Putrycz
paper_authors._id: 65cc2d6a11a80579a430d71c
paper_authors.name: Soledad López Gambino
paper_authors._id: 65cc2d6a11a80579a430d71d
paper_authors.name: Kayeon Yoo
paper_authors._id: 65cc2d6a11a80579a430d71e
paper_authors.name: Elena Sokolova
paper_authors._id: 65cc2d6a11a80579a430d71f
paper_authors.name: Thomas Drugman
paper_title: BASE TTS: Lessons from building a billion-parameter Text-to-Speech model
    on 100K hours of data
paper_summary: We introduce a text-to-speech (TTS) model called BASE TTS, which stands for
  Big Adaptive Streamable TTS with
  Emergent abilities. BASE TTS is the largest TTS model to-date,
  trained on 100K hours of public domain speech data, achieving a new
  state-of-the-art in speech naturalness. It deploys a 1-billion-parameter
  autoregressive Transformer that converts raw texts into discrete codes
  ("speechcodes") followed by a convolution-based decoder which converts these
  speechcodes into waveforms in an incremental, streamable manner. Further, our
  speechcodes are built using a novel speech tokenization technique that features
  speaker ID disentanglement and compression with byte-pair encoding. Echoing the
  widely-reported "emergent abilities" of large language models when trained on
  increasing volume of data, we show that BASE TTS variants built with 10K+ hours
  and 500M+ parameters begin to demonstrate natural prosody on textually complex
  sentences. We design and share a specialized dataset to measure these emergent
  abilities for text-to-speech. We showcase state-of-the-art naturalness of BASE
  TTS by evaluating against baselines that include publicly available large-scale
  text-to-speech systems: YourTTS, Bark and TortoiseTTS. Audio samples generated
  by the model can be heard at https://amazon-ltts-paper.com/.
paper_ai_summary: BASE TTS, a large-scale text-to-speech model, demonstrates advanced naturalness and emergent abilities using autoregressive Transformers and novel speech tokenization techniques.
paper_ai_keywords: autoregressive Transformer speechcodes convolution-based decoder speech tokenization speaker ID disentanglement byte-pair encoding natural prosody