Multimodal gesture recognition based on Choquet integral

Author

Hirota, K. ; Vu, H.A. ; Le, P.Q. ; Fatichah, C. ; Liu, Z. ; Tang, Y. ; Tangel, M.L. ; Mu, Z. ; Sun, B. ; Yan, F. ; Masano, D. ; Thet, O. ; Yamaguchi, M. ; Dong, F. ; Yamazaki, Y.

Author_Institution

Dept. of Comput. Intell. & Syst. Sci., Tokyo Inst. of Technol., Yokohama, Japan

fYear

2011

fDate

27-30 June 2011

Firstpage

772

Lastpage

776

Abstract

A multimodal gesture recognition method is proposed based on Choquet integral by fusing information from camera and 3D accelerometer data. By calculating the optimal fuzzy measures for the camera recognition module and the accelerometer recognition module, the proposal obtains enough recognition rate 92.7% in average for 8 types of gestures by improving the recognition rate approximate 20% compared to that of each module. The proposed method aims to realize the casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.

Keywords

fuzzy set theory; gesture recognition; integral equations; 3D accelerometer data; Choquet integral; accelerometer recognition module; camera recognition module; information fusion; multimodal gesture recognition; nonverbal gesture message; optimal fuzzy measures; recognition rate; verbal message; Accelerometers; Cameras; Gesture recognition; Humans; Robot sensing systems; Training data; 3D Accelerometer; Choquet Integral; Gesture Recognition; Human-Robot Interaction; Sensor Fusion;

fLanguage

English

Publisher

ieee

Conference_Titel

Fuzzy Systems (FUZZ), 2011 IEEE International Conference on

Conference_Location

Taipei

ISSN

1098-7584

Print_ISBN

978-1-4244-7315-1

Electronic_ISBN

1098-7584

Type

conf

DOI

10.1109/FUZZY.2011.6007382

Filename

6007382