Fusing appearance and geometric constraints for estimating the epipolar geometry

Recovering the epipolar geometry of a stereo image pair is important for many computer and robotic vision systems, for performing motion recovering, 3D reconstruction and, more recently, image retrieval from large databases. Most state-of-the-art methods for estimating the fundamental matrix rely solely in putative image correspondences, and, therefore, heavily depend on the capability of the low-level image features to provide enough distinctiveness capabilities for establishing correct matches. In this paper we present a robust method for estimating the fundamental matrix based on all image features, and not only matching points. This is done by selecting the best correspondent keypoints between views through a proper weighting function that fuses local appearance of keypoints and distance to the epipolar lines. Several distance weighting functions are compared, with an intuitive theoretical analysis of the role of each function parametrization being analyzed. Experimental evidence shows that our approach outperforms the current state-of-the-art methods in terms of error magnitude, number of correct matches provided and computational time.