5

M2UGen

A PREPRINT

4.1 MUCaps Dataset

We develop the MUCaps dataset which is composed
of text-music pairs, encompassing approximately 1,200
hours of music sourced from AudioSet [22] and some
publicly accessible music websites. We leverage the MU-
LLaMA model to caption the music files. For the cap-
tioning process, the MU-LLaMA model is presented with
each music file along with the following question: “De-
scribe the music in detail, including aspects such as in-
struments used, tempo, and the mood of the song”. The
generated MUCaps dataset is employed for the purpose of
encoder and decoder alignment training.

4.2 MUEdit Dataset

To empower our model with the capability of performing
music editing in response to prompts, we have curated the
MUEdit dataset, which includes 55.69 hours of 10-second
music pairs. The generation of this dataset is carried out
through the following steps:

1) Generate captions for all the music files acquired using

the MU-LLaMA model.
M2UGen