generated MIDI in pianoroll format, together with two selected video scenes (sky

and dancing) from different times in the video. Notably, for the scene depicting

the sky, there is a discernible pattern of low loudness and note density levels in

the generated music. In contrast, during the dancing scene, the figure illustrates

a distinctive increase in both loudness and note density. This demonstrates that

our model’s ability to dynamically adapt music to match specific video content.

38

Figure 14: Generated MIDI in pianoroll together with estimated loudness (dur-

ing postprocessing).

7. Conclusion

In conclusion, our work represents a significant stride in the field of multi-

modal generative systems, by introducing a novel Video2Music framework for

generating music that seamlessly matches accompanying videos. This is a novel

task, which will be further facilitated by our development of MuVi-Sync, a
Video2Music