Intelligent invariance techniques for music gesture recognition based on skin modelling

A computer vision methodology for the recognition of finger gestures performed on a music instrument has been recently developed and implemented in the PianOrasis system. PianOrasis recognises the gestures of all the five fingers simultaneously, but not in real-time. In this paper, an optimisation of the above methodology is presented, implying the recognition of finger musical gestures performed in space. Scale and rotation invariance techniques are integrated into the system increasing the recognition quality and reducing the processing time. Scale invariance rests on the deterministic modelling of the number of skin pixels in the image. The proposed modelling enable three different sets of filtering parameters for the hand segmentation process, overcoming a long and manual preliminary analysis. More flexible finger gestures, performed in space without music instrument, can be recognised because of the integration of the rotation invariance.