Highly accurate video object identification utilizing hint information

We propose a hint-information based object identification system for video to significantly improve the object recognition accuracy of the conventional object identification system. To this end, we first formulate a novel cost function to ensure good local representation and good content variation coverage of candidate key frames. We then apply dynamic programming on the cost function to extract key frames from the input video to summarize and represent the whole video. Finally, we recognize the objects in the key frames using the learned model on the conventional knowledge database (i.e., training images) and use these labeled recognized objects as hint information to refine the knowledge database. The good representativeness of hint information alleviates large variations between training and testing images and therefore significantly improves the object recognition performance. As a proof of concept, we use face identification as an example to demonstrate the effectiveness of the proposed hint-information based object identification system. Our extensive experimental results on three types of movies demonstrate the important role of hint information in the proposed system and the excellent performance of the proposed system when compared to the conventional object identification system without using hint information.