Unscented feature tracking

Accurate feature tracking is the foundation of many high level tasks in computer vision, such as 3D reconstruction and motion analysis. Although there are many feature tracking algorithms, most of them do not maintain information about the error of the data being tracked. Also, due to the difficulty and spatial locality of the problem, existing methods can generate grossly incorrect correspondences, making outlier rejection an essential post-processing step. We propose a new generic framework that uses the Scaled Unscented Transform to augment arbitrary feature tracking algorithms, and use Gaussian Random Variables (GRV) for the representation of features’ locations uncertainties. We apply and validate the framework on the well-understood Kanade–Lucas–Tomasi feature tracker, and call it Unscented KLT (UKLT). The UKLT tracks GRVs and rejects incorrect correspondences, without a global model of motion. We validate our method on real and synthetic sequences, and demonstrate how the UKLT outperforms other approaches on both outlier rejection and the accuracy of feature locations.