Benchmarking superpixel descriptors

Superpixels are useful intermediate representations for many computer vision tasks. While the segmentation step is well studied, the subsequent creation of meaningful descriptors lacks this foundation. Superpixels have similar properties like affine covariant regions (keypoints), but there are fundamental differences that led to a different set of commonly used descriptors. In this paper we work towards general insights on requirements and properties of superpixel descriptors as well as a framework for experimental comparison. More precisely, we want to answer the question: Given superpixels from different images, what can superpixel descriptors tell about the ground truth overlap of the segments in the world? We propose and discuss an evaluation methodology based on image sequences with ground truth optical flow. Further, we present results of several types of superpixel descriptors and discuss the influence of the used segmentation algorithm as well as the problem of visual ambiguity in oversegmentations.