Recognizing emotions from videos by studying facial expressions, body postures and hand gestures

A system for recognizing emotions from videos by studying facial expressions, hand gestures and body postures is presented. A stochastic context-free grammar (SCFG) containing 8 combinations of hand gestures and body postures for each emotion is used and we show that increasing the number of combinations in SCFG improves the system´s generalization for new hand gesture and body posture combinations. We show that hand gestures and body postures contribute to improving the emotion recognition rate with up to 5% for Anger, Sadness and Fear compared to the standard facial emotion recognition system, while for Happiness, Surprise and Disgust no significant improvement was noticed.