Quantitative evaluation of rank-order similarity of images

Region importance maps from image understanding algorithms and human observer studies are ordered rankings of the pixel locations. Kemeny and Snell´s distance (d_KS), an existing measure from ordinal ranking theory, can thus be used as a similarity measure between images. We address three problems with d_KS: its high computational cost, its bias in favor of images with sparse histograms, and its image-size dependent range of values. We present a novel computationally efficient algorithm for computing d_KS between two images, and we derive a normalized form d_KS with no bias whose range is independent of image size. For evaluating an algorithm where the reference data and algorithm output are ordered rankings of pixels, d_KS is subjectively superior to the correlation coefficient as a figure of merit