  A Music Video (MV) is a video aiming at visually illustrating or extending
the meaning of its background music. This paper proposes a novel method to
automatically generate, from an input music track, a music video made of
segments of Youtube music videos which would fit this music. The system
analyzes the input music to find its genre (pop, rock, ...) and finds segmented
MVs with the same genre in the database. Then, a K-Means clustering is done to
group video segments by color histogram, meaning segments of MVs having the
same global distribution of colors. A few clusters are randomly selected, then
are assembled around music boundaries, which are moments where a significant
change in the music occurs (for instance, transitioning from verse to chorus).
This way, when the music changes, the video color mood changes as well. This
work aims at generating high-quality realistic MVs, which could be mistaken for
man-made MVs. By asking users to identify, in a batch of music videos
containing professional MVs, amateur-made MVs and generated MVs by our
algorithm, we show that our algorithm gives satisfying results, as 45% of
generated videos are mistaken for professional MVs and 21.6% are mistaken for
amateur-made MVs. More information can be found in the project website:
http://ml.cs.tsinghua.edu.cn/~sarah/

Multimedia