Distilling classification models from cross validation runs: an application to mass spectrometry

We present work on a proteomics application. More specifically, from the domain of mass-spectrometry and the identification of biomarkers for stroke attacks. Mass-spectrometry based biomarker identification is an application that sets a number of challenges to the knowledge discovery process. We describe how we tackle them and present a number of machine learning experiments that we performed in order to identify the most suitable learning algorithm for the given problem. However working with real world applications one of the main issues apart from good classification performance is an indication of the factors that really determine the classification decision. Usually based on the results of a resampled-based performance estimation, e.g. cross validation, an algorithm is selected that will provide the operational classification model. On a next step the operational model should be constructed, nevertheless it is not obvious how this should be done since in resampled-based procedures a number of different models are created. We propose a method for linear classifiers that examines the different models produced with cross-validation. The method examines the stability of the models produced from the different training folds and combines them to provide a single model.