Baselines We consider three baselines: 1) VALL-E [Wang et al., 2023], SOTA for English zero-shot
TTS trained on Librilight. 2) YourTTS [Casanova et al., 2021], SOTA multilingual (English, French,
and Portuguese) zero-shot TTS model trained on VCTK, LibriTTS, TTS-Portugese [Casanova et al.,
2022], and M-AILABS French. It is a flow-based model adapted from VITS [Kim et al., 2021]
using a pre-trained multilingual speaker embedder for voice conditioning. 3) A3T [Bai et al., 2022],
SOTA for NAR speech editing and infilling trained with a regression loss on VCTK. We also consider
Demucs [Défossez et al., 2020], a SOTA speech enhancement model trained with regression and
adversarial losses for denoising experiments. Table 1 summarizes the tasks each baseline is capable
of solving.
Voicebox-Text-GuidedMultilingual UniversalSpeechGenerationatScale