Robust object tracking using color and depth images with a depth based occlusion handling and recovery

Object tracking has always been a challenging problem in the field of computer vision. Most previous works used only the color images for this problem, which may suffer from changing environments, such as illumination variations, partial occlusion and background clutters, resulting in the socalled model drift problem. As depth images contain spatial information, depth images encode important clues for these problems. In this paper, we utilize depth images, which are provided by the Microsoft Kinect, to improve the traditional tracking methods. Firstly, we adopt the harr-like features extracted from both the color and depth domains with a naive Bayesian classifier to reduce model drift. Secondly, to handle the occlusion problem, we utilize the depth histogram to decide whether occlusion occurs. Thirdly, to recover from heavy occlusions, we design a detection module to search for the occluded object and use the depth-based segmentation to find the object. We carry out experiments on the Princeton RGBD tracking dataset, and the results demonstrate the effectiveness of the proposed method, especially under occlusion conditions.