Simultaneous Detection of Multiple Facial Action Units via Hierarchical Task Structure Learning

Automatic facial action unit (AU) detection is a challenging research topic in computer vision and pattern recognition. Most of the existing approaches design classifiers to detect AUs individually without considering their intrinsic relations. This paper proposes a novel framework to jointly learn the classifiers for detecting the presence and absence of multiple AUs. In our method, hierarchical structure is defined to model the relations among a set of AU detection tasks, where each leaf denotes a specific AU. The relatedness among AUs is captured by introducing a latent layer whose nodes represent the common properties across several subsets of AUs. Multi-task multiple kernel learning (MTMKL) approach is utilized to simultaneously learn the similarities between AUs within our hierarchical model and the SVM discriminative hyper plane for detecting each AU. Extensive experiments on the CK+ and DISFA databases show that by exploiting the AU inter-relations, our proposed method has achieved encouraging performance on AU detection compared to several state-of-the-art methods.