Simultaneous tracking and segmentation of two free moving hands in a video conferencing scenario

This paper discusses an efficient and effective approach to simultaneously identifying and tracking two free moving and gesturing hands, together with the provision of accurate segmentation masks of the hands regions. It is aimed at a novel application in telepresence video conferencing system. The skin colour model in YUV colour space is created and adapted to characterise the likely appearance of hands. The initialisation is performed on the first two frames of a video stream, and the probabilistic skin colour map and motion mask information are used, resulting in an accurate segmentation of the two initial hands. The ensued tracking uses a set of Kalman filters to predict the centroid, orientation, and sizes of candidate hand objects, from which refinements of hands locations are made. The issue of inter-hands overlap and update of the skin colour model are also properly addressed.