How many reconstruction methods are needed for training a numerical observer?

In medical imaging it is now established that image quality should be evaluated using task-based criteria, such as human-observer (HO) performance in a medical decision task (e.g. lesion-detection). HO studies are usually costly and time consuming, therefore the development of a numerical observer (NO) surrogate, an algorithm that mimics HO, is highly desirable. Recently, we proposed and successfully tested a supervised-learning approach for modeling HO with a machine-learning algorithm (namely a support vector machine). In the supervised-learning approach, the goal is to identify the mapping (regression) between measured image features and defect likelihood scores given to an image by an HO. To identify this mapping (training phase), the proposed methodology uses a number of images for which human observer scores are available. The number of images and reconstruction methods for which the HO scores are available are limited. Therefore, in this work we are evaluating the proposed machine-learning based numerical observer performance as a function of the number of different reconstruction methods used during the training phase. The results indicate, as would be expected, that the more reconstruction methods used, the better the NO performance, but, surprisingly, the improvement of having more than five or six reconstruction methods is not significant.