A hybrid approach to news video classification multimodal features

This paper presents a hybrid approach to the classification of news video story. Most of current works on news story classification utilize the multimodal features in a uniform manner. However, the reliability of audio-visual confidence is much lower than that of text, which may evidently lower-down the performance of the classification. We proposed a decision strategy mainly depends on the evidence from text classifiers with extra assistance of audio-visual clues. In our approach, SVMs for text features and GMMs for audio-visual features are first built for each category and then used to compute text and audio-visual confidence vectors respectively. To make final decision, a text-biased decision strategy is proposed to combine these multimodal confidence vectors. To validate the performance, text-based classification and SVM-based meta-classification methods are compared on large-scale news stories from TV programs, and our proposed hybrid approach achieves the best overall performance.