Silhouette measurements for Bayesian object tracking in noisy point clouds

In this paper, we consider the problem of jointly tracking the pose and shape of objects based on noisy data from cameras and depth sensors. Our proposed approach formalizes object silhouettes from image data as measurements within a Bayesian estimation framework. Projecting object silhouettes from images back into space yields a visual hull that constrains the object. In this work, we focus on the 2D case. We derive a general equation for the silhouette measurement update that explicitly considers segmentation uncertainty of each pixel. By assuming a bounded error for the silhouettes, we can reduce the complexity of the general solution to only consider uncertain edges and derive an approximate measurement update. In simulations, we show that the proposed approach dramatically improves point-cloud-based estimators, especially in the presence of high noise.