Stochastic Discriminant Analysis

In this paper, we consider a linear supervised dimension reduction method for classification settings: Stochastic Discriminant Analysis. This method matches point similarities in the projection space with those in a response space. These similarities are represented by t-distributed joint pairwise probabilities. The matching is done by minimizing the Kullback-Leibler divergence between the two probability distributions. The performance of the algorithm is compared against state-of-the-art methods in supervised dimension reduction. We found that the performance of SDA is comparable to (and sometimes better than) state-of-the-art methods in supervised linear dimension reduction. In the presence of multiple classes, low-dimensional SDA projections led to higher classification accuracies.