Track based relevance feedback for tracing persons in surveillance videos

While surveillance cameras are a powerful tool for the prevention, detection and resolving of crimes, for most cases the amount of video data has become unmanageable. To ease the analysis, various automatic methods have been proposed, focusing on data-management, detecting suspicious behavior, person recognition, or event reconstruction. In this paper we focus on event reconstruction, in particular on tracing the whereabouts of people. The standard approach for such event reconstruction is to first detect persons in single frames and then match a query to all detections to retrieve the same person in multiple cameras. However, since the number of detected persons is large and performance of matching techniques limited, this process is slow and prone to errors. Intelligent interactive techniques are urged for. We propose to represent detected persons by their complete track within a single camera instead of a single detection and thereby reduce the search-space. On these tracks we use Relevance Feedback to improve recall with only a small effort of the user. Testing the tracking method on a benchmark dataset and a real-life dataset led to a reduction of the search space of 90%, while tracing accuracy based on the distance between tracks improved recall by up to 110% when compared to random tracing. Adding Relevance Feedback led to an additional improvement in recall of up to 400% compared to sequential scanning using the same number of visual assessments.