Principal appearance and motion from boosted spatiotemporal descriptors

Feature definition and selection are two important aspects in visual analysis of motion. In this paper, spatiotemporal local binary patterns computed at multiple resolutions are proposed for describing dynamic events, combining static and dynamic information from different spatiotemporal resolutions. Appearance and motion are the key components for visual analysis related to movements. AdaBoost algorithm is utilized for learning the principal appearance and motion from spatiotemporal descriptors derived from three orthogonal planes, providing important information about the locations and types of features for further analysis. In addition, learners are designed for selecting the most important features for each specific pair of different classes. The experiments carried out on diverse visual analysis tasks: facial expression recognition and visual speech recognition, show the effectiveness of the approach.