A single model mastering all these aspects would
make a strong contribution to the music industry,
as it can enable the broader public to be part of
the creative process by allowing them to compose
music using an accessible text-based interface, as-
sist creators in finding inspiration, and provide an
unlimited supply of novel audio samples.
To address these challenges, we propose Moûsai,2
a novel text-conditional two-stage cascading diffu-
sion model. Specifically, the first stage trains a mu-
sic encoder by diffusion magnitude-autoencoding
(DMAE), which compress audio by the novel dif-
fusion autoencoder; and the second stage learns to
generate the reduced representation while condi-
tioning on a textual description by text-conditioned
latent diffusion (TCLD). The two-stage generation
process is shown in Figure 1.
MOUSAI