The role of image understanding in contour detection

Many cues have been proposed for contour detection or image segmentation. These include low-level image gradients to high-level information such as the identity of the objects in the scene or 3D depth understanding. While state-of-the-art approaches have been incorporating more cues, the relative importance of the cues is unclear. In this paper, we examine the relative importance of low-, mid- and high-level cues to gain a better understanding of their role in detecting object contours in an image. To accomplish this task, we conduct numerous human studies and compare their performance to several popular segmentation and contour detection machine approaches. Our findings suggest that the current state-of-the-art contour detection algorithms perform as well as humans using low-level cues. We also find evidence that the recognition of objects, but not occlusion information, leads to improved human performance. Moreover, when objects are recognized by humans, their contour detection performance increases over current machine algorithms. Finally, mid-level cues appear to offer a larger performance boost than high-level cues such as recognition.