A Failure-Aware Explicit Shape Regression Model for Facial Landmark Detection in Video

Facial landmark detection is fundamental for various face-related applications such as interactive avatars on mobile devices. Awareness of detection failure is critical for practical applications because even occasional failures to detect facial landmarks lead to bad user experience. This letter proposes a fast and robust AdaBoost Based Cascade Detector (ABCD) for discerning failures from shape regression in video on mobile devices. A vector of randomly sampled pixel intensities near facial landmarks is taken as the input feature for AdaBoost classifiers. Several AdaBoost classifiers are cascaded together for robustness, computational efficiency and to augment the theoretical number of false samples in training. With this failure detector, the correctly estimated shape of the previous frame can be utilized in the next frame for initialization, which not only improves the regression accuracy but also saves on face searching time. ABCD is incorporated into a recently proposed facial landmark detection algorithm Face Alignment by Explicit Shape Regression (FAESR). Experiments on videos show that failure awareness powered FAESR yields an accurate and automatic facial landmark detection with very low computational costs, which is suitable for real time application on mobile devices.