The present invention relates generally to video summarization and more particularly to relevancy in video summarization.
Video summarization may be used to present the main theme of video sequences to a user. This summarization is especially useful if the volume of the video is large and the content is highly redundant.
As technology advances, just about any digital device is capable of capturing video. Examples of these videos include raw footage of a short event (e.g., a child's first words, etc.), a series of re-takes to capture a single event to be edited in production, or a single long relatively static recording (e.g., surveillance video, etc.). To assist users in selecting and extracting interesting and/or relevant parts of these videos, summarization systems and methods are used. Summaries are shorter than the original video source and usually aim to capture only small parts of the content as important events or scenes, such as the utterance of a phrase, a person, an action, or a setting. To compose a summary, several events or scenes are arranged in a way that is easy for the user to understand. In this way, the original meaning and even context of the events is not lost.
Conventional applications make use of video summarization for generating home videos, assisting film editors in finding the best scene “take” from a series of shoots, and assisting video analysis in an intelligence community. One particular type of videos, “rushes,” are generally used to produce professional programming, such as television dramas. Rushes are raw, unedited video. Rushes are unique because they generally contain many repeated recordings of a single event that has been altered in some way (e.g., a person's dialog has changed, their position in the scene is slightly different, etc.).
Since rushes are captured for subsequent editing, they generally contain large numbers of repeated scenes and ideas. For example, actors in a television drama may be asked to repeat a particular scene or action multiple times so that video editors and directors have multiple “takes” to choose from. Additionally, this video also contains non-essential (e.g., “junk”) content such as color bars to signal the starting of a new recording session and calibrate equipment or physical clappers used to guarantee synchronization between the audio and video streams and provide metadata (e.g., date, series name, scene or take information, etc.) about the next recorded scene. The creation of a rush video summary involves many as yet unrealized goals: reduce junk content, identify and reduce the count of repeated ideas, prioritize and choose only the most interesting ideas, and present the summarized set of ideas in a comprehensible and logical manner.
Accordingly, improved systems and methods for video summarization are required.