Minimizing neighborhood evidential decision error for feature evaluation and selection based on evidence theory

Feature selection is an important preprocessing step in pattern recognition and machine learning, and feature evaluation arises as key issues in the construction of feature selection algorithms. In this study, we introduce a new concept of neighborhood evidential decision error to evaluate the quality of candidate features and construct a greedy forward algorithm for feature selection. This technique considers both the Bayes error rate of classification and spatial information of samples in the decision boundary regions. the decision boundary regions, each sample xi in the neighborhood of x provides a piece of evidence reflecting the decision of x so as to separate the decision boundary regions into two subsets: recognizable and misclassified regions. The percentage of misclassified samples is viewed as the Bayes error rate of classification in the corresponding feature subspaces. By minimizing the neighborhood evidential decision error (i.e., Bayes error rate), the optimal feature subsets of raw data set can be selected. Some numerical experiments were conducted to validate the proposed technique by using nine UCI classification datasets. The experimental results showed that this technique is effective in most of the cases, and is insensitive to the size of neighborhood comparing with other feature evaluation functions such as the neighborhood dependency.