Indexing Faces in Broadcast News Video Archives

Face indexing and retrieval are basic tasks of search engines. Most current search engines use text information such as keywords and captions rather than visual content for indexing. This approach returns many irrelevant results, since faces and names are not usually aligned in video data. We propose an unsupervised framework for indexing faces in video archives of broadcast news. First, the faces in all video frames are detected, and the detected faces within one video shot are grouped into face tracks so that each track consists of different facial expressions of one individual. In this way, the appearances of persons in different shots can be identified by matching face tracks rather than single faces. Second, the face tracks are grouped into clusters and a list of candidate clusters for each name is created by observing co-occurrences of face tracks and names. Each cluster might correspond to several names. To associate each cluster with only one name, we minimize the intra-variance of clusters. Given a name as a query, we use a text-based search to return a list of candidate face tracks. Then for each face track, we compute the distances to the clusters associated with the query name. The distances are used to rank the face tracks. Experiments on the TRECVID 2003 dataset consisting of 120 hours of video prove the effectiveness of our approach.