From Table 2, we can see that CoDi performs the worst
of all the models for the task of text-to-music generation.
This is likely due to the fact that it has not been trained
on music data. As mentioned above, our model incorpo-
rates AudioLDM 2 and MusicGen as music decoders, so
in Table 2, M2UGen v1 is compared to AudioLDM 2, and
M2UGen v2 is compared to MusicGen. It can be observed
that our M2UGen model performs better when given Au-
dioLDM 2 or MusicGen as the music decoder compared
to using them alone. Particularly, the generated music is
more relevant to the input instructions, as indicated by the
improvement in CLAP score. This can be attributed to
the use of LLMs, which makes it easier for the model to
understand the input instructions and use them to guide
music generation.

7

M2UGen

A PREPRINT

5.3 Prompt Based Music Editing

5.4 Multi-modal Music Generation
M2UGen