Correlation and MSVM-based feature selection

A central problem in machine learning is identifying a representative set of features from which we would construct a classification model for a particular task. This paper addresses the problem of feature selection for machine learning through a correlation and MSVM (modified support vector machines) based approach. The central hypothesis is that a good feature set contains features that are highly correlated with the class, yet uncorrelated with each other. So we introduce the CMFS (correlation and MSVM-based feature selection). First, CMFS ranks the features using MSVM according to their correlation with the class. Secondly, CMFS uses a forward selection search with correlation-based method to form feature subset. A feature can be added to the feature set or not decided by the class separability of the feature and the correlation with the already chosen features. Experiments on artificial and natural datasets show that, compared with other algorithms, CMFS typically eliminates well much more features with less time and higher accuracy.