Face Tracking and Facial Feature Detection with a Webcam

We first present a multiple faces tracking algorithm using color. The skin color distribution is evaluated over the face candidate resulting from Viola&Jones face detector. This face detector is robust in case of an up-right face, but fails to detect slanted faces (pan and tilt pose): location and size of face are tracked using the skin color distribution evaluated on the last face sub-image detected. Facial features are searched over faces candidates using a structural approach. Edge orientations of the face is computed for facial landmarks detection. Central symmetry of eyebrow, mouth and eye shapes is used by a cumulative method close to the Hough transform and referred to as Chinese Transform. The complete system runs at 15 frames/s.