Creating motion video summaries with partial audio-visual alignment

In this paper, we propose an audio-visual summarization system which creates an audio and a visual summary of a given video program separately, and then integrates the two summaries with a partial alignment. A bipartite graph-based audio-visual alignment algorithm is developed to efficiently find the best alignment solution that satisfies the predefined alignment requirements. With the proposed system, we strive to produce a motion video summary for the original video that: (1) provides a natural visual and audio content overview; and (2) maximizes the coverage for both audio and visual contents of the original video without having to sacrifice either of them. Such audio-visual summaries dramatically increase the information intensity and depth, and lead to a more effective video content overview.