The application of Efron´s bootstrap methods in validating feature classification using artificial neural networks for the analysis of mammographic masses

Efron´s bootstrap resampling method is used to analyze the performance of artificial neural networks (ANNs) in the area of feature classification for the analysis of mammographic masses. The purpose of feature classification in mammography is to discover the salient information that can be used to discriminate benign from malignant masses. The performance of ANNs is typically measured in terms of the area under the receiver operating characteristics (ROC) curve (A_z). Performance uncertainty problems and the generalization problems of ANNs are still the critical issues that impede the further application of ANNs in clinical medicine. It is unreasonable and impractical to justify the performance of one ANN being better than another just by its best A_z value. Efron´s bootstrap methods make it possible to quantitatively analyze the performance of ANNs and anticipate its change tendency with relatively high accuracy. Our experimental results show that the probability model of A_z is close to a normal distribution. The performance of ANNs is more sensitive to the change of topology than that of the size and the composition of the training set. Bootstrap methods can be used to find the optimal epochs and avoid overfitting.