Text-To-Speech Synthesis In The Wild
<|reference_start|>Text-To-Speech Synthesis In The Wild: Text-to-speech (TTS) systems are traditionally trained using modest databases of studio-quality, prompted or read speech collected in benign acoustic environments such as anechoic rooms. The recent literature nonetheless shows efforts to train TTS systems using data collected in the wild. While this approach allows for the use of massive quantities of natural speech, until now, there are no common datasets. We introduce the TTS In the Wild (TITW) dataset, the result of a fully automated pipeline, in this case, applied to the VoxCeleb1 dataset commonly used for speaker recognition. We further propose two training sets. TITW-Hard is derived from the transcription, segmentation, and selection of VoxCeleb1 source data. TITW-Easy is derived from the additional application of enhancement and additional data selection based on DNSMOS. We show that a number of recent TTS models can be trained successfully using TITW-Easy, but that it remains extremely challenging to produce similar results using TITW-Hard. Both the dataset and protocols are publicly available and support the benchmarking of TTS systems trained using TITW data.<|reference_end|>
@article{jung2024text-to-speech,
    title={Text-To-Speech Synthesis In The Wild},
    author={Jee-weon Jung and Wangyou Zhang and Soumi Maiti and Yihan Wu and Xin
  Wang and Ji-Hoon Kim and Yuta Matsunaga and Seyun Um and Jinchuan Tian and
  Hye-jin Shim and Nicholas Evans and Joon Son Chung and Shinnosuke Takamichi
  and Shinji Watanabe},
    journal={arXiv preprint arXiv:2409.08711},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.08711},
    primaryClass={eess.AS cs.AI}
}