Learning of demonstrated grasping skills by stereoscopic tracking of human head configuration

In this paper a novel approach to learning by demonstration (LbD) is presented. A multimodal service robot is taught grasping skills by a human instructor who demonstrates a grasping action. Our approach contributes novel solutions to the aspects of robustly tracking the demonstrator´s hands in real time as well as to the transformation of tracking results into grasping skills. To track the demonstrator´s hands in stereoscopic images a mean-shift-like algorithm is adapted. For the very first time this algorithm is applied to local binary patterns (LBP) and color histograms. To retrieve the hand configuration we use view-based principal component analysis (PCA). To develop grasping skills from tracking results the robot repetitively tracks the demonstrator´s grasping actions and transforms the results into three-dimensional self organizing maps (SOMs). The SOMs give a spatial description of the collected data and serve as data structures for a reinforcement learning (RL) algorithm which optimizes trajectories for use by the robot. The approach is applied to a multimodal service robot. Experiments show the effectiveness of the LBP-enhanced mean-shift-like tracking and the robustness of LbD based on SOMs and RL