An alternative approach is using diffusion models. Schneider et al. [2023], Huang et al. [2023b],
Maina [2023], Forsgren and Martiros proposed using a latent diffusion model for the task of
text-to-music. Schneider et al. [2023] proposed using diffusion models for both audio encoder-decoder
and latent generation. Huang et al. [2023b] proposed a cascade of diffusion model to generate
audio and gradually increase its sampling rate. Forsgren and Martiros proposed fine-tuning Stable
Diffusion Rombach et al. [2022] using spectrograms to generate five-second segments, then, using
image-to-image mapping and latent interpolation to generate long sequences.
Audio generation. Several studies were proposed for text-to-audio (environmental sounds) genera-
tion. Yang et al. [2022] proposed representing audio spectrograms using a VQ-VAE, then applying a
discrete diffusion model conditioned on textual CLIP embeddings for the generation part [Radford
Simple and Controllable Music Generation