Automatic Object Classification through Semantic Analysis

Currently available methods for object recognition and classification primarily rely on static information in single-frame images. However, for the combat aerial video (usually low resolution video), all these static indexes used for object classification and recognition are almost impossible to obtain. To address this challenge, we propose an innovative 3D and dynamic semantic scene analysis based approach that exploits surveillance video data mainly captured from UAV platforms to classify static object (e.g. buildings) and moving object (e.g. vehicles) automatically. In our proposed automatic object detection and classification framework, in addition to 3D static object´s visual features (e.g. building´s or vehicle´s shape, line orientation, color, and texture) and the 3D static structures of the urban environment, we also explore dynamic video features which include vehicle motion patterns over time. All these static and dynamic features will be considered to construct spatial-temporal feature vectors, and the new generated vectors will then be sent to a probabilistic dynamic influence diagram (DID) reasoning model for real-time and automatic building and vehicle classification. In addition, we also propose novel 3D algorithms on automatic building detection, 3D terrain modeling, and visualization to support accurate object categorization/classification.