  In recent years, the burgeoning interest in diffusion models has led to
significant advances in image and speech generation. Nevertheless, the direct
synthesis of music waveforms from unrestricted textual prompts remains a
relatively underexplored domain. In response to this lacuna, this paper
introduces a pioneering contribution in the form of a text-to-waveform music
generation model, underpinned by the utilization of diffusion models. Our
methodology hinges on the innovative incorporation of free-form textual prompts
as conditional factors to guide the waveform generation process within the
diffusion model framework. Addressing the challenge of limited text-music
parallel data, we undertake the creation of a dataset by harnessing web
resources, a task facilitated by weak supervision techniques. Furthermore, a
rigorous empirical inquiry is undertaken to contrast the efficacy of two
distinct prompt formats for text conditioning, namely, music tags and
unconstrained textual descriptions. The outcomes of this comparative analysis
affirm the superior performance of our proposed model in terms of enhancing
text-music relevance. Finally, our work culminates in a demonstrative
exhibition of the excellent capabilities of our model in text-to-music
generation. We further demonstrate that our generated music in the waveform
domain outperforms previous works by a large margin in terms of diversity,
quality, and text-music relevance.

Sound;Artificial Intelligence;Computation and Language;Multimedia;Audio and Speech Processing