On multi-task learning for facial action unit detection

In this paper we investigate the use of Multitask Learning (MTL) methods to model the commonalities and variations across a set of facial action units (AUs) and also learn the classifiers for detection of multiple AUs simultaneously by exploiting their inner-relations. We studied three variants of MTL algorithms, the Regularized MTL (RMTL), the Multitask Feature Learning (MTFL) and the Alternating Multi-task Structure Learning (AMTSL). We used two databases to evaluate the performance of the MTL methods; the first one is the extended Cohn-Kanade (CK+) database with posed AUs while the second is the DISFA database consisting of spontaneous AUs. Compared with the canonical Support Vector Machine (SVM) which detects AUs individually without considering their relationships, the MTL-based methods show significant improvements in the F1 reliability measurement. In particular, the RMTL algorithm consistently outperforms the other investigated MTL-based classifiers as well as several state-of-the-art methods on the CK+ database while for spontaneous AUs on the DISFA database the MTFL approach achieves the best performance.