to the timesteps 9/8 sec to 16/8 sec.

4.2.3. Velocity estimation based on loudness

We recognize that the music’s volume should synchronize with the emotional

intensity and visual dynamics of the video. To achieve this synchronization, we

convert the predicted loudness based on the video features (with the model

described above), into a parameter known as MIDI velocity, which governs the

perceived loudness of the notes in the music. The conversion is achieved through

a linear mapping procedure, where the predicted loudness values, ranging from

0 to 1, are translated into corresponding MIDI velocity values within the defined

29

MIDI velocity range of 49 to 112.

By establishing a connection between loudness levels and visual features, we

hope to forge a cohesive link between the auditory and visual elements. As the

video becomes more intense, the music can respond by growing louder, and as

the video becomes more tranquil, the music may adopt a softer demeanor.
Video2Music