Method for qualitatively evaluating CVIR algorithms based on human similarity judgments

It is almost impossible to quantitatively evaluate the performance of CVIR (content based visual information retrieval) at present. A qualitative method for the task may be better. CVIR is subjected to computer vision field. The ground truth of CVIR benchmark must be calibrated by real users. Therefore, the theory of computer vision and some assumptions inferred from perceptual psychophysics can be used to qualitatively evaluate the CVIR methods. In this paper, we first introduce our idea of the qualitative evaluation method. Then we will summarize some of important conclusion of psychophysics and corresponding physical constraints really adopted by human vision. Finally, some examples are introduced, which show that our method work reasonably well.