BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data
<|reference_start|>BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data: We introduce a text-to-speech (TTS) model called BASE TTS, which stands for $\textbf{B}$ig $\textbf{A}$daptive $\textbf{S}$treamable TTS with $\textbf{E}$mergent abilities. BASE TTS is the largest TTS model to-date, trained on 100K hours of public domain speech data, achieving a new state-of-the-art in speech naturalness. It deploys a 1-billion-parameter autoregressive Transformer that converts raw texts into discrete codes ("speechcodes") followed by a convolution-based decoder which converts these speechcodes into waveforms in an incremental, streamable manner. Further, our speechcodes are built using a novel speech tokenization technique that features speaker ID disentanglement and compression with byte-pair encoding. Echoing the widely-reported "emergent abilities" of large language models when trained on increasing volume of data, we show that BASE TTS variants built with 10K+ hours and 500M+ parameters begin to demonstrate natural prosody on textually complex sentences. We design and share a specialized dataset to measure these emergent abilities for text-to-speech. We showcase state-of-the-art naturalness of BASE TTS by evaluating against baselines that include publicly available large-scale text-to-speech systems: YourTTS, Bark and TortoiseTTS. Audio samples generated by the model can be heard at https://amazon-ltts-paper.com/.<|reference_end|>
@article{łajszczak2024base,
    title={BASE TTS: Lessons from building a billion-parameter Text-to-Speech model
  on 100K hours of data},
    author={Mateusz {L}ajszczak, Guillermo C'ambara, Yang Li, Fatih Beyhan,
  Arent van Korlaar, Fan Yang, Arnaud Joly, 'Alvaro Mart'in-Cortinas, Ammar
  Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszy'nska, Haohan
  Guo, Bartosz Putrycz, Soledad L'opez Gambino, Kayeon Yoo, Elena Sokolova,
  Thomas Drugman},
    journal={arXiv preprint arXiv:2402.08093},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.08093},
    primaryClass={cs.LG cs.CL eess.AS}
}