Title :
What are they doing? : Collective activity classification using spatio-temporal relationship among people
Author :
Choi, Wongun ; Shahid, Khuram ; Savarese, Silvio
Author_Institution :
Univ. of Michigan, Ann Arbor, MI, USA
fDate :
Sept. 27 2009-Oct. 4 2009
Abstract :
In this paper we present a new framework for pedestrian action categorization. Our method enables the classification of actions whose semantic can be only analyzed by looking at the collective behavior of pedestrians in the scene. Examples of these actions are waiting by a street intersection versus standing in a queue. To that end, we exploit the spatial distribution of pedestrians in the scene as well as their pose and motion for achieving robust action classification. Our proposed solution employs extended Kalman filtering for tracking of detected pedestrians in 2D 1/2 scene coordinates as well as camera parameter and horizon estimation for tracker filtering and stabilization. We present a local spatio-temporal descriptor effective in capturing the spatial distribution of pedestrians over time as well as their pose. This descriptor captures pedestrian activity while requiring no high level scene understanding. Our work is tested against highly challenging real world pedestrian video sequences captured by low resolution hand held cameras. Experimental results on a 5-class action dataset indicate that our solution: i) is effective in classifying collective pedestrian activities; ii) is tolerant to challenging real world conditions such as variation in illumination, scale, viewpoint as well as partial occlusion and background motion; iii) outperforms state-of-the art action classification techniques.
Keywords :
Kalman filters; image classification; video signal processing; action classification technique; background motion; collective activity classification; extended Kalman filtering; partial occlusion; pedestrian action categorization; pedestrian activity; pedestrian detection; pedestrian video sequences; robust action classification; spatial distribution; spatio-temporal descriptor; spatio-temporal relationship; street intersection; tracker filtering; tracker stabilization; Cameras; Filtering; Kalman filters; Layout; Lighting; Parameter estimation; Robustness; Spatial resolution; Testing; Video sequences;
Conference_Titel :
Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on
Conference_Location :
Kyoto
Print_ISBN :
978-1-4244-4442-7
Electronic_ISBN :
978-1-4244-4441-0
DOI :
10.1109/ICCVW.2009.5457461