Generalization evaluation of numerical observers for image quality assessment

It is now widely accepted that image quality should be evaluated by using task-based criteria, such as human-observer (HO) performance in a lesion-detection task. Because a HO study is typically costly and time consuming, development of a numerical observer (NO) surrogate is highly desirable. Recently we proposed a supervised-learning approach for modeling HO with a numerical observer, where a machine-learning algorithm is used to model the relationship between measured image features and HO scores. In this work, we further develop and evaluate this approach. We develop a NO in a lesion-detection task using a channelized support vector machine (CSVM), and compare its performance against the widely used channelized Hotelling observer (CHO). In particular we employ a cross-validation procedure to quantify the generalization ability of a NO. It allows us to evaluate how well a NO can perform when it is trained on one set of images, but tested on a different set of images. Our evaluation results demonstrate that while both CHO and CSVM can generalize well when training and test images are both reconstructed in the same way, the CSVM can generalize better when the test images are reconstructed in a different way from that of training images.