Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation
<|reference_start|>Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation: Audio token modeling has become a powerful framework for speech synthesis, with two-stage approaches employing semantic tokens remaining prevalent. In this paper, we aim to simplify this process by introducing a semantic knowledge distillation method that enables high-quality speech generation in a single stage. Our proposed model improves speech quality, intelligibility, and speaker similarity compared to a single-stage baseline. Although two-stage systems still lead in intelligibility, our model significantly narrows the gap while delivering comparable speech quality. These findings showcase the potential of single-stage models to achieve efficient, high-quality TTS with a more compact and streamlined architecture.<|reference_end|>
@article{gállego2024single-stage,
    title={Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge
  Distillation},
    author={Gerard I. G'allego and Roy Fejgin and Chunghsin Yeh and Xiaoyu Liu
  and Gautam Bhattacharya},
    journal={arXiv preprint arXiv:2409.11003},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.11003},
    primaryClass={cs.SD cs.AI eess.AS eess.SP}
}