Face tracking using Rao-Blackwellized particle filter and pose-dependent probabilistic PCA

This paper deals with tracking of face blobs containing pose changes. We propose a novel tracking method to deal with face pose changes during the tracking. In the method, tracking is formulated as an approximate solution to the MAP estimate of the state vector, consisting of a linear and a nonlinear part. Multi-pose face appearances are described by local linear models, each being related to a single pose and estimated by probabilistic PCA (PPCA). A Markov model with pose indices as its states is used to model the transitions between poses. Shape and locations of face blobs and the associated pose indices are assumed to be nonlinear, and are estimated by a Rao-Blackwellized particle filter (RBPF). This enables a separate estimation of the linear state vector through marginalizing the joint probability. The proposed method has been tested for videos containing frequent face pose changes and large illumination variations, where 5 poses (left, frontal, right, up, down) were modeled. The tracking results are shown to be robust to variable speed of pose changes and with relatively tight boxes.