Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
<|reference_start|>Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation: Diffusion models have achieved remarkable success in text-to-speech (TTS), even in zero-shot scenarios. Recent efforts aim to address the trade-off between inference speed and sound quality, often considered the primary drawback of diffusion models. However, we find a critical mispronunciation issue is being overlooked. Our preliminary study reveals the unstable pronunciation resulting from the diffusion process. Based on this observation, we introduce StableForm-TTS, a novel zero-shot speech synthesis framework designed to produce robust pronunciation while maintaining the advantages of diffusion modeling. By pioneering the adoption of source-filter theory in diffusion TTS, we propose an elaborate architecture for stable formant generation. Experimental results on unseen speakers show that our model outperforms the state-of-the-art method in terms of pronunciation accuracy and naturalness, with comparable speaker similarity. Moreover, our model demonstrates effective scalability as both data and model sizes increase.<|reference_end|>
@article{han2024improving,
    title={Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via
  Stable Formant Generation},
    author={Changjin Han, Seokgi Lee, Gyuhyeon Nam, Gyeongsu Chae},
    journal={arXiv preprint arXiv:2409.09311},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.09311},
    primaryClass={eess.AS cs.SD}
}