FLY-TTS: Fast, Lightweight and High-Quality End-to-End Text-to-Speech Synthesis
<|reference_start|>FLY-TTS: Fast, Lightweight and High-Quality End-to-End Text-to-Speech Synthesis: While recent advances in Text-To-Speech synthesis have yielded remarkable improvements in generating high-quality speech, research on lightweight and fast models is limited. This paper introduces FLY-TTS, a new fast, lightweight and high-quality speech synthesis system based on VITS. Specifically, 1) We replace the decoder with ConvNeXt blocks that generate Fourier spectral coefficients followed by the inverse short-time Fourier transform to synthesize waveforms; 2) To compress the model size, we introduce the grouped parameter-sharing mechanism to the text encoder and flow-based model; 3) We further employ the large pre-trained WavLM model for adversarial training to improve synthesis quality. Experimental results show that our model achieves a real-time factor of 0.0139 on an Intel Core i9 CPU, 8.8x faster than the baseline (0.1221), with a 1.6x parameter compression. Objective and subjective evaluations indicate that FLY-TTS exhibits comparable speech quality to the strong baseline.<|reference_end|>
@article{guo2024fly-tts:,
    title={FLY-TTS: Fast, Lightweight and High-Quality End-to-End Text-to-Speech
  Synthesis},
    author={Yinlin Guo, Yening Lv, Jinqiao Dou, Yan Zhang, Yuehai Wang},
    journal={arXiv preprint arXiv:2407.00753},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.00753},
    primaryClass={eess.AS cs.SD}
}