Regular simplex criterion: A novel feature extraction criterion

Feature extraction is an important topic in machine learning. There are two representative criterions for feature extraction, i.e. Fisher Criterion and Maximum Margin Criterion. In this paper, we propose a new criterion, called Regular Simplex Criterion. This criterion requires that samples from the same class are projected to the same point, while samples from different classes have unit distance. Under this criterion, we present a novel dimensionality reduction method, namely Linear Simplex Analysis (LSA). LSA is solved by multivariate linear regression with a specific definition of class indicator matrix which has a strong geometrical interpretation, i.e. each column of this matrix corresponds to a vertex of a regular simplex. Several variants of LSA, e.g. Regularized Simplex Analysis (RSA) and Kernel Simplex Analysis (KSA), are also proposed. Encouraging experimental results on UCI machine learning database indicate that the new criterion as well as the proposed methods are very effective.