chitecture for Chord Transformer to learn the long-term de-
pendency of input video feature sequences. The event-based
token sequence is added with positional encoding and fed
into the Chord Transformer as a query. Meanwhile, the style
features from video controller are fed as keys and values.
Melody Transformer. We employ an encoder-decoder trans-
former architecture for melody generation. The encoder
receives a chord sequence as input, and then the decoder
generates a note sequence as the output melody. Considering
the relatively short-range dependency between melody and
chords, we adopt a bar-level cross-attention mask so that
each decoder token can only attend to the contextual encoder
output within the previous current or next bar.
Accompaniment Transformer Similarly, we also adopt an
encoder-decoder transformer to generate the accompaniment
sequence. Since accompaniment closely correlates with
chords and melody, we merge the generated chord sequence
VideoBackgroundMusicGeneration