sion. The Transformer Decoder’s goal is to predict a sequence of Chord events
Outputchord ∈ RC×L where L is the vocabulary size of chord events. At each
time step, the Decoder takes as input the previously generated feature encoding

over the chord event vocabulary and the visual features, to predict the next

chord event.

23

In contrast to the standard Transformer model, which uses positional si-

nusoids for timing information, we incorporate relative position representa-

tions (Shaw et al., 2018). These representations explicitly encode the dis-

tance between tokens in a sequence, a crucial consideration for music appli-

cations (Huang et al., 2018). We adopt a strategy similar to Huang et al. (2018)

to jointly learn ordered relative position embeddings R for all possible pairwise

distances among pairs of query and key positions within each attention head:

Attnrelative(Q, K, V ) = softmax

(cid:18) QK T + R√

(cid:19)

V

Dk
Video2Music