When Occlusions are Outliers

In many tracking applications, the deformable object of interest suffers from frequent occlusions. Traditional augmenting methods use templates and measures of similarity to recover from occlusions. In this paper, we break with these methods. Instead, we model bad image correspondences, which are induced by occlusions, as statistical outliers in the context of tracking high-dimensional deformable models. This interpretation allows us to use robust statistical estimators in the deformable model’s parameter space to detect and eliminate such outliers. Because fast-moving occlusions can generate an excessively large outlier to inlier ratio in the occluded areas, we combine the robust statistical estimation with an initial rejection of correspondences based on the magnitude of the optical flow, a simple 2D criterion. To improve robustness even further, we have the final outlier rejection test take into account both the statistical distribution of the deformable model’s parameters, and that different parameters are affected by different subsets of correspondences. We validate and demonstrate our technique on real sequences of American Sign Language, which exhibit frequent and extensive occlusions caused by fast movement of the subjects’ hands.