2.5D Facial Expression Recognition using Photometric Stereo and the Area Weighted Histogram of Shape Index

In this work we argue that the high frequency spatial variations in the topological information of the face are important for Facial Expression Recognition. Stereo and laser scanner based datasets currently used are inherently regularized, resulting in the loss of high frequency information. We test our hypothesis on the dense gradient field from Photometric Stereo which preserves this high frequency information. To overcome the geometric artefacts introduced through the integration of the gradient field we take a local approach and, assuming piecewise smoothness, we directly extract the second order differential geometry. We introduce the Area Weighted Histogram of Shape Index which is invariant to both scale and orientation and extend this to a localized histogram approach. Rather than using heuristically chosen areas of the face we use a data driven approach based on the Fisher Discriminant Ratio to identify the most discriminatory regions of the face. Using a non-linear Support Vector Machine we are able to recognize the six prototypic expressions of the face. We carry out analysis on the Binghamton BU4DFE database as well as a small Photometric Stereo dataset and show that the high frequency information preserved by Photometric Stereo may be highly useful for automatic Facial Expression Recognition.