5.2 Neural Speech Synthesis
5.2.1 Task Description
Neural speech synthesis is a technology that utilizes artificial intelligence and deep learning
techniques to create speech from text or other inputs. Its applications are widespread, including
in healthcare, where it can be used to develop assistive technologies for those who are unable to
communicate due to neurological impairments. To generate speech, deep neural networks like
CNNs, RNNs, transformers, and diffusion models are trained using phonemes and the mel spectrum.
The process involves several components, such as text analysis, acoustic models, and vocoders, as
shown in Figure 14. Acoustic models convert linguistic features into acoustic features, which are
then used by the vocoder to synthesize the final speech signal. Various architectures, including
neural vocoders based on GANs like HiFi-GAN [268], are used by the vocoder to generate speech.
AReviewofDeepLearningTechniquesforSpeechProcessing