Performance validation of microarray analysis methods

Following the rapid development of gene selection methods, several comparison studies have been reported for ranking methods on various datasets. In order to reduce bias in performance measures, most studies use an evaluation scheme based on cross-validation. In this paper we focus on the methodology of evaluation itself and address methodological problems using three representative algorithms on two public datasets. More specifically, the paper discusses the need of an independent test-set to reduce bias associated with cross-validation, the use of case specific considerations for generalization, as well as other measures that reflect stability and consistency of the result. Such measures reflect the influence of the actual dataset distribution on the performance of gene selection methods.