Genetic programming for performance improvement and dimensionality reduction of classification problems

In this paper, Genetic programming (GP) is used to construct a new set of high level features based on the original attributes of a classification problem with the goal of improving the classification performance and reducing the dimensionality. A non-wrapper approach is taken and a new fitness function is proposed based on the Renyi entropy. The GP system uses a variable terminal pool which is constructed by the class-wise orthogonal transformations of the original features. The performance measure is classification accuracy on 12 benchmark problems using constructed features in a decision tree classifier. The performance over difficult problems has been improved by constructing features for compound classes. This approach is compared with the principle component analysis (PCA) method and the results show that the new approach outperforms the PCA method on most of the problems in terms of classification performance and dimensionality reduction.