Subspace selection using semi-supervised harmonic mean of Kullback-Leibler divergences

In many areas of pattern recognition and machine learning, subspace selection is an essential step. Fisher´s linear discriminant analysis (LDA) is one of the most well-known linear subspace selection methods. However, LDA suffers from the class separation problem. The projection to a subspace tends to merge close class pairs. A recent result, named maximizing the geometric mean of Kullback-Leibler (KL) divergences of class pairs (MGMD), can significantly reduce the class separation problem. Furthermore, maximizing the harmonic mean of Kullback-Leibler (KL) divergences of class pairs (MHMD) emphasizes smaller divergences more than MGMD, and deals with the class separation problem more effectively. However, in many applications, labeled data are very limited while unlabeled data can be easily obtained. The estimation of divergences of class pairs is unstable using inadequate labeled data. To take advantage of unlabeled data for subspace selection, semi-supervised MHMD (SSMHMD) is proposed using graph Laplacian as normalization. Quasi-Newton method is adopted to solve the optimization problem. Experiments on synthetic data and real image data show the validity of SSMHMD.