Hybrid Approach for Facial Feature Detection and Tracking under Occlusion

When a face is partially occluded in an image, the existing discriminative or generative methods often do not find facial features. This is due to the limitations of local facial feature detectors and appearance modeling in discriminative and generative methods, respectively. To solve this problem, we propose a new facial feature detection method that hybridizes the discriminative and generative methods. The proposed method consists of an initialization stage and optimization stage. The initialization stage detects the face, estimates the facial pose, and obtains the initial parameter set by locating the pose-specific mean shape on the detected face. The optimization stage obtains the facial features by updating the parameter set using the combined Hessian matrix and gradient vector of shape and appearance errors obtained from two methods. Further, we extend the proposed facial feature detection to face tracking by adding a template face obtained from the previous image frame. In experiments, the proposed method yields more accurate facial feature detection or tracking under heavy occlusions and pose variations than the existing methods.