Face tracking system based on color, stereovision and elliptical shape features

We present a vision system that tracks a human face in 3D. We combine color and stereo cues to find likely image regions where a face may exist. A greedy search algorithm examines for a face candidate, focusing action around the position at which the face was detected in the previous time step. The aim of the search is to find the best-fit head ellipse. The size of the searched ellipse projected into the image is scaled depending on the depth information. The final position of the ellipse is determined on the basis of intensity gradient near the edge of the ellipse, depth gradient along the head boundary and matching of the color histograms representing the interior of the actual and the previous ellipse. The color histogram and parameters of the ellipse are dynamically updated over time and compared with previous ones. The frontal view face is detected using PCA to make the tracking more reliable and, in particular, to update the color model over time with only face-like skin pixels.