We train three stages separately and connect them to
form a complete pipeline during inference. We construct
transformers [49] based on linear transformer [26] to reduce
time consumption. All three stages are trained with cross-
entropy loss and teacher-forcing strategy. During inference,
we use a stochastic temperature-controlled sampling [19]

to increase the diversity of generated samples. We train
Chord, Melody, and Accompaniment Transformers for 200,
200, and 400 epochs, respectively, on one V100 GPU. We
use fluidsynth3 to synthesize our MIDI into audio. More
implementation details are in the Appendix.
5. Evaluation Metric

In this chapter, we first extend the vision-language
CLIP [36] to the video-music domain and propose a
new evaluation metric named Video-Music CLIP Precision
(VMCP) to measure the video-music correspondence.
5.1. Video-Music CLIP
VideoBackgroundMusicGeneration