A two-population evolutionary algorithm for feature extraction: Combining filter and wrapper

Extracting good features is critical to the performance of learning algorithms such as classifiers. Feature extraction selects and transforms original features to find information hidden in data. Due to the huge search space of selection and transformation of features, exhaustive search is computationally prohibitive and randomized search such as evolutionary algorithms (EA) are often used. In our prior work on evolutionary-based feature extraction, an individual, which represents a set of features, is evaluated by estimating the accuracy of a classifier when the individual´s feature set is used for learning. Although incorporating a learning algorithm during evaluation, which is called the wrapper approach, generally performs better than evaluating an individual simply by the statistical properties of data, which is called the filter appproach, our EA based on a wrapper approach suffers from overfitting, so that a slight enhancement of fitness in training can dramatically reduce the classification accuracy for unseen testing data. To cope with this problem, this paper proposes a two-population EA for feature extraction (TEAFE) that combines filter and wrapper approaches, and shows the promising preliminary results.