Polish sign language words recognition with Kinect

The paper considers Polish sign language (PSL) words recognition with sensor Kinect. The nearest neighbour classifier with dynamic time warping technique was examined. The classifier was using two sets of features, the first produced by Kinect (in a form of 3D positions of most important joints of observed person body - a skeletal image or a skeleton) and the second describing hands, which were tracked as skin colour regions in the images acquired by Kinect. Obtained feature vectors representing PSL words are clustered in order to discover natural data divisions among them. This step is helpful in revealing possible problems which could occur during recognition and provides additional information about processed data. Results of ten-fold cross-validation tests for the classifier with two feature sets are given. The classifier with data provided by Kinect yielded promising recognition accuracy (89.33%) but in order to obtain better results (98.33%) adding new features responsible for hand shape description should be considered.