Feature Extraction Algorithm Based on K Nearest Neighbor Local Margin

Feature extraction is the transformation of high-dimensional data into a meaningful representation of reduced dimensionality. The representation extracted are often beneficial to mitigate the computational complexity and improve the accuracy of a particular classifier. In this paper we introduce a novel feature extraction algorithm called K nearest neighbor local margin maximization and apply it to measure the quality of the reduced features in the context of supervised classification problems. Using the concept of the hypothesis margin, we aim to find a discriminant subspace in which each projected point is well separated from the affine hull of its K local nearest neighbors. The experimental results on three high dimensional data sets demonstrate the effectiveness of our algorithm.