On linear dimension reduction for multiclass classification of Gaussian mixtures

Linear dimension reduction (LDR) offers a computationally attractive approach to feature extraction. Specifically for classification, numerous methods of LDR have been proposed. Many LDR algorithms implicitly or explicitly assume the Gaussian data model to derive LDR criteria. Gaussian mixture models (GMMs) offer the accuracy and flexibility that can be achieved with nonparametric models as well as the advantage of parametric representation. In this paper, we develop an LDR method that is applicable for multiclass classification of GMMs. This method is based on a novel upper bound we derive for the classification error rate of GMMs in terms of the Chernoff distance between the Gaussian mixture components. We then present a gradient descent algorithm to minimize the bound with respect to the LDR transformation. The resulting linear transformation achieves higher classification rates than the classical methods and is competitive to the state-of-the-art LDR methods.