3.2.3. Scene offset

We used the PySceneDetect library (Castellano, 2018) to accurately detect

shot changes in videos. Figure 7 shows a few examples of the scene detection

results. Instead of directly utilizing scene IDs as a feature to incorporate scene

change information, we chose to calculate scene offsets based on the detected

scene IDs. By introducing a scene offset value that initiates at 0 and progres-

sively increments until the next scene change, we effectively capture the relative

position of each frame within a scene. This approach allows us to implicitly en-

code the scene change information by considering the temporal distance between

17

Figure 6: Examples of the CLIP probability results for each of the six emotion

frames within and across scenes.

classes.

3.2.4. Motion

To estimate motion or changes in the visual content of the video, we first

computed the RGB Difference between the current frame and the preceding
Video2Music