Classification and feature selection with human performance data

We investigate the utility of a novel form of prior, namely the accuracies with which humans categorize briefly displayed images. Such information reflects the complexity of an image for the visual system and carries information about the features important for categorization. We incorporate the prior in an SVM framework, by biasing the decision boundary towards examples difficult for humans, and by learning a suitable kernel. We focus on the task indoors vs. outdoors using a variety of histogram and interest point features. We observe improvement in classification especially for the indoor class when gist features are used.