Cross-Dialect Text-To-Speech in Pitch-Accent Language Incorporating Multi-Dialect Phoneme-Level BERT
<|reference_start|>Cross-Dialect Text-To-Speech in Pitch-Accent Language Incorporating Multi-Dialect Phoneme-Level BERT: We explore cross-dialect text-to-speech (CD-TTS), a task to synthesize learned speakers' voices in non-native dialects, especially in pitch-accent languages. CD-TTS is important for developing voice agents that naturally communicate with people across regions. We present a novel TTS model comprising three sub-modules to perform competitively at this task. We first train a backbone TTS model to synthesize dialect speech from a text conditioned on phoneme-level accent latent variables (ALVs) extracted from speech by a reference encoder. Then, we train an ALV predictor to predict ALVs tailored to a target dialect from input text leveraging our novel multi-dialect phoneme-level BERT. We conduct multi-dialect TTS experiments and evaluate the effectiveness of our model by comparing it with a baseline derived from conventional dialect TTS methods. The results show that our model improves the dialectal naturalness of synthetic speech in CD-TTS.<|reference_end|>
@article{yamauchi2024cross-dialect,
    title={Cross-Dialect Text-To-Speech in Pitch-Accent Language Incorporating
  Multi-Dialect Phoneme-Level BERT},
    author={Kazuki Yamauchi, Yuki Saito, Hiroshi Saruwatari},
    journal={arXiv preprint arXiv:2409.07265},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.07265},
    primaryClass={cs.SD cs.CL eess.AS}
}