Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis
<|reference_start|>Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis: Recent language model-based text-to-speech (TTS) frameworks demonstrate scalability and in-context learning capabilities. However, they suffer from robustness issues due to the accumulation of errors in speech unit predictions during autoregressive language modeling. In this paper, we propose a phonetic enhanced language modeling method to improve the performance of TTS models. We leverage self-supervised representations that are phonetically rich as the training target for the autoregressive language model. Subsequently, a non-autoregressive model is employed to predict discrete acoustic codecs that contain fine-grained acoustic details. The TTS model focuses solely on linguistic modeling during autoregressive training, thereby reducing the error propagation that occurs in non-autoregressive training. Both objective and subjective evaluations validate the effectiveness of our proposed method.<|reference_end|>
@article{zhou2024phonetic,
    title={Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis},
    author={Kun Zhou, Shengkui Zhao, Yukun Ma, Chong Zhang, Hao Wang, Dianwen Ng,
  Chongjia Ni, Nguyen Trung Hieu, Jia Qi Yip, Bin Ma},
    journal={arXiv preprint arXiv:2406.02009},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.02009},
    primaryClass={eess.AS cs.CL cs.SD}
}