Figure 1: Multi-modal music understanding and generation by our proposed M2UGen framework.

ABSTRACT

The current landscape of research leveraging large lan-
guage models (LLMs) is experiencing a surge. Many
works harness the powerful reasoning capabilities of these
models to comprehend various modalities, such as text,
speech, images, videos, etc. They also utilize LLMs
to understand human intention and generate desired out-
puts like images, videos and music. However, research
that combines both understanding and generation us-
ing LLMs is still limited and in its nascent stage. To

†Corresponding author.
M2UGen