DensityRank: A novel feature ranking method based on kernel estimation

This paper proposes a novel feature ranking method, DensityRank, based on kernel estimation on the feature spaces to improve the classification performance. As the availability of raw data in many of today´s applications continues to grow at an explosive rate, it is critical to assess the learning capabilities of different features and select the important subset of features to improve learning accuracy as well as reduce computational cost. In our approach, kernel methods are used to estimate the probability density function for each feature across different class labels. Discrepancy analysis based on the mean integrated square error (MISE) between pairs of such density estimations is used to provide the ranking values. Then, the ranked subspace method is adopted to select subsets of important features that are used to develop the learning models. Comparative study of this method with those of traditional ranking methods related to Fisher´s discrimination ratio and information gain theory, as well as the random subspace algorithm and the bootstrap aggregating (bagging), are presented in this paper. Simulation results on various real-world data sets illustrate the effectiveness of the proposed method.