Hypotheses for Image Features, Icons and Textons

We review ideas about the relationship between qualitative description of local image structure and quantitative description based on responses to a family of linear filters. We propose a sequence of three linking hypotheses. The first, the Feature Hypothesis, is that qualitative descriptions arise from a category system on filterresponse space. The second, the Icon Hypothesis, is that the partitioning into categories of filter response space is determined by a system of iconic images, one associated with each point of the space. The third, the Texton Hypothesis, is that the correct images to play the role of icons are those that are the most likely explanations of a vector of filter responses. We present results in support of these three hypotheses, including new results on 2-D 1st order structure