Structure-from-Motion reconstruction based on weighted Hamming descriptors

We propose a pipelined methods to reduce memory consumption of large-scale Structure-from-Motion reconstruction with the use of unsorted images extracted from photo collection websites. Recent research is able to reconstruct cities based on extracted images from photo collection websites. SIFT feature is used to find the correspondences between two images. For the large-scale reconstruction with unsorted images, the system needs to store all the descriptors and feature points information in memory to search for correspondences. As each SIFT descriptor is a 128 dimensional real-value vector, storing all the descriptors would consume a significant amount of memory. Based on this limitation, we project the high dimensional features into a low-dimensional space using a learned projection matrix. After projection, the distance of the descriptors belonging to the same point in 3D space is decreased; the distance of the descriptors belonging to the different points is increased. Furthermore, we learn a mapping function, which maps the real-value descriptor into binary code. As Hamming descriptors contain only two value options per bit and the length of the descriptor is limited, there are usually multiple descriptors having the same Hamming distance to the query descriptor. In dealing with this problem, we give different weights to each dimension and rank each bit of the Hamming descriptor based on each dimensions discriminant power; this contributes to reduce the ambiguity in matching the descriptors. The experiments show that our method achieves dense reconstruction results with less than 10 percent of the original memory consumption.