Structurally refined feature matching for multi-source images

In this paper we consider the problem of matching feature points between multi-source images. Straightforwardly comparing visual features such as SIFT may result in desirable local feature correspondences for single-source images. However, when it comes to matching feature between multi-source images, e.g. between an RGB color image and an infrared image, the feature based matching scheme tends to yield certain mismatches. To address this shortcoming, we develop a structural constraint for refining the multi-source feature matching performance. Our structural refinement rejects the small number of mismatches which contradict the spatial consistency that the large number of correct matches follow. The key to the effectiveness of our method is the observation that correct matches turn out to form a tight cluster in a subspace spanned in the light of the spatial relationships of feature points; on the other hand, the mismatches are generally located far away from the cluster as outliers in the subspace. Experimental results validate that our method not only properly refines the feature correspondences between multi-source images, but also outperforms alternative state-of-the-art cross-field matching methods.