Person attribute search for large-area video surveillance

This paper describes novel video analytics technology which allows an operator to search through large volumes of surveillance video data to find persons that match a particular attribute profile. Since the proposed technique is geared for surveillance of large areas, this profile consists of attributes that are observable at a distance (including clothing information, hair color, gender, etc) rather than identifying information at the face level. The purpose of this tool is to allow security staff or investigators to quickly locate a person-of-interest in real time (e.g., based on witness descriptions) or to speed up the process of video-based forensic investigations. The proposed algorithm consists of two main components: a technique for detecting individual moving persons in large and potentially crowded scenes, and an algorithm for scoring how well each detection matches a given attribute profile based on a generative probabilistic model. The system described in this paper has been implemented as a proof-of-concept interactive software tool and has been applied to different test video datasets, including collections in an airport terminal and collections in an outdoor environment for law enforcement monitoring. This paper discusses performance statistics measured on these datasets, as well as key algorithmic challenges and useful extensions of this work based on end-user feedback.