Real-time facial feature extraction and emotion recognition

In this paper, the implementation details of a real-time facial feature extraction and emotion recognition system are discussed. The proposed method uses edge counting and image-correlation optical flow techniques to calculate the local motion vectors of facial feature. Thereafter the determination of the emotional state of a subject using a neural network is discussed. The main objective of the paper is the real-time implementation of a facial emotion recognition system. The techniques used in conventional off-line calculations are modified to make real-time implementation possible. A variety of improved techniques to the general algorithms such as edge focusing, global motion cancellation are proposed and implemented. In this paper, a maximum frame rate of 11.1 per second for a resolution of 320 by 240 is achieved. Feature extraction accuracy as high as 93.3% for best combination facial feature extraction was achieved. Emotion recognition accuracy as high as 60% was achieved based on the Carnegie Mellon University (CMU) face database. Speed improvement achieved over previous off-line processing method is in excess of 20 times.