Establishing Correspondences between Planar Image Patches

Finding correspondences between image pairs is a fundamental task in computer vision. Herein, we focus on establishing matches between images of urban scenes which are typically composed of planar surface patches with highly repetitive structures. The latter property makes traditional point-based methods unreliable. The basic idea of our approach is to formulate the correspondence problem in terms of homography estimation between planar image regions: given a planar region in one image, we are simultaneously looking for its corresponding segmentation in the other image and the planar homography acting between the two regions. We will show, that due to the overlapping views the general 8 degree of freedom (DOF) of the homography mapping can be geometrically constrained to 3 DOF and the resulting segmentation/registration problem can be efficiently solved by finding the region´s occurrence in the second image using pyramid representation and normalized mutual information as the intensity similarity measure. The method has been validated on a large database of building images taken by different mobile cameras and quantitative evaluation confirms robustness against intensity variations, occlusions or the presence of non-planar parts. We also show examples of 3D planar surface reconstruction as well as 2D mosaicking.