A novel technique to acquire perceived utility scores from textual descriptions of distorted natural images

Many applications value an assessment of distorted natural images according to their usefulness, or utility, rather than their perceptual quality. For the quality task, human observers evaluate an image based on its perceptual resemblance to a reference, whereas for the utility task, the usefulness of an image as a surrogate for a reference is under evaluation. This paper presents a novel technique for acquiring perceived utility scores derived from textual descriptions produced by observers viewing images. The technique uses an observer-centric approach, so observers dictate the relevant concepts that characterize image usefulness. This technique is used to collect perceived utility (PU) scores for 150 distorted images that simulate scenes captured by a surveillance system. The capability of both the natural image contour evaluation (NICE) utility estimator, which compares contours of the reference and test images, and popular quality estimators to estimate PU is reported. The conclusions drawn from the results augment previously reported results and establish that a multi-scale implementation of NICE (MS-NICE) is the most robust utility estimator among the estimators evaluated, since MS-NICE consistently performs as well as estimators producing the most accurate perceived utility estimates for various distortion types.