Gesture recognition using hybrid generative-discriminative approach with Fisher Vector

Gesture recognition is used for many practical applications such as human-robot interaction, medical rehabilitation and sign language. In this paper, we apply a hybrid generative-discriminative approach by using the Fisher Vector to improve the recognition performance. The strategy is to merge the generative approach of Hidden Markov Model dealing with spatio-temporal motion data with the discriminative approach of Support Vector Machine focusing on the classification task. The motion segments are encoded into HMMs, and each segment is converted to FV, whose elements can be obtained as the derivative of the probability of the segment being generated by the HMMs with respect to their parameters. SVM is subsequently trained by the FVs. An input gesture can be classified to corresponding gesture category by SVM. In the experiments, we test our approach by comparing three HMM chain models and four categorization methods on dataset provided by the ChaLearn Looking at People Challenge 2014 (LAP 2014). The results show that similar gesture patterns are clustered closely in several categories. Our approach based left-to-right HMMs outperforms other gesture recognition methods. More specifically, the hybrid generative-discriminative approach overcomes the standard HMM approach and the generative kernel approach overcomes the generative embedding approach. For these results, our approach is effective to improve the recognition performance.