A Peak Detection Method for Understanding User States for Empathetic Intelligent Agents

Recognition of facial expression is a useful and unobtrusive means for machines to understand users´ internal states. However, most human facial expression is ambiguous or subtle to recognize resulting in poor accuracy. To overcome this limitation, we propose a peak detection method to select only meaningful images from image sequences which imply significant changes of facial expression by calculating differences between images. We believe this method will alleviate the accuracy degradation caused by different personal appearances and ambiguous facial expressions. When applied to commercial products, it can provides a suitable method for adaptation of the empathetic agent embedded in machines such as personal assistants on TV, smartphone and vehicles based on recognized facial expression of users. For experimental validation of the proposed method, we tested four different features for measuring image similarity with the extended Cohn-Kanade facial image dataset. As a result, we could get better recognition accuracy than the original image sequences. Moreover, we reduced the number of images that need to be recognized to 24.52% without degradation of accuracy.