A framework for real time gesture recognition for interactive mobile robots

Interactive mobile robots are an active area of research. The paper presents a framework for designing a real-time vision based hand-body gesture recognition system for such robots. The said framework works in real world lighting conditions, with complex backgrounds, and can handle intermittent motion of the camera. We present a novel way in which the MHI (motion history images) and MEI (motion energy images) are built. The input signal is captured by a singular monocular color camera. Vision is the only feedback sensor used. It is assumed that the gesturer is wearing clothes that are slightly different from the background. The motion and color cues are combined in a robust way. Gestures are first learned offline and then matched to the temporal data generated online in real time. We have tested this on a gesture database consisting of 11 hand-body gestures and have recorded recognition accuracy up to 90%.