Sparse coding-based co-salient object detection with application to video abstraction

Automatic abstraction is important for video retrieval and browsing in a semantic manner. Detecting the focus of interest such as co-occurring objects in video frames automatically can benefit the tedious manual labelling process. However, detecting the co-occurring objects that is visually salient in video sequences is a challenging task. In this paper, in order to detect co-salient video objects efficiently, we first use the preattentive scheme to locate the co-salient regions in video frames and then measure the similarity between salient regions based on KL-divergence. In addition, to update preattentive patch set for co-salient objects, sparse coding is used for dictionary learning and further discrimination among co-salient objects. Finally a set of primary co-salient objects can be found across all video frames using our proposed filtering scheme. As a result, a video sequence can be automatically parsed based on the detection of co-occurring video objects. Our experiment results show that the proposed co-salient video objects modeling achieves high precision value about 85% and reveals its robustness and feasibility in videos.