Distance metric learning with penalized linear discriminant analysis

Linear discriminant analysis has gained extensive applications in supervised classification and dimension reduction. In LDA formulation, original patterns with high dimension can be projected to lower dimension through a transfer matrix which is fundamental to clustering, nearest neighbor searches, and others. The transfer matrix is usually viewed as a distance metric. However, the classification accuracy under the LDA metric is neither optimal nor suboptimal because physical datasets often appear multimodal distribution. This paper proposes a penalized scheme for LDA to improve the classification rate by using the information of misclassified samples. This method is evaluated to be robust and effective by a great number of datasets from the machine learning repository.