Interactive person re-identification in TV series

In this paper, we present a system for person re-identification in TV series. In the context of video retrieval, person re-identification refers to the task where a user clicks on a person in a video frame and the system then finds other occurrences of the same person in the same or different videos. The main characteristic of this scenario is that no previously collected training data is available, so no person-specific models can be trained in advance. Additionally, the query data is limited to the image that the user clicks on. These conditions pose a great challenge to the re-identification system, which has to find the same person in other shots despite large variations in the person´s appearance. In the study, facial appearance is used as the re-identification cue. In order to increase the amount of available face data, the proposed system employs a face tracker that can track faces up to full profile views. A fast and robust face recognition algorithm is used to find matching faces. If the match result is highly confident, our system adds the matching face track to the query set. This approach help to increase the variation in the query set, making it possible to retrieve results with different poses, illumination conditions, etc. Furthermore, if the user is not satisfied with the number of returned results, the system can present a small number of candidate face images and lets the user confirm the ones that belong to the queried person. The system is extensively evaluated on two episodes of the TV series Coupling, showing very promising results.