Laplacian Eigenmaps Latent Variable Model modification for pattern recognition

Laplacian Eigenmaps Latent Variable Model (LELVM) is a probabilistic dimensionality reduction model that combines the advantages of latent variable models and observed variables, applied to many practical problems such as pattern recognition. Non-linear dimensionality reduction techniques are affected by two critical aspects: (1) the design of the adjacency graphs, and (2) the embedding of new test data - the out-of-sample problem. For the first aspect, we modify graph construction by changing LE objective function. We add an entropy term to LE objective function. In this way, we obtain a principled edge weight updating formula which naturally corresponds to classical heat kernel weights. For the second aspect, we use the sparse representation approach as a solution to the `out-of-sample´ problem. The proposed method is simple, non-parametric and computationally inexpensive. Experimental result on UCI datasets using different classifiers show the feasibility and effectiveness of the proposed method in comparison to conventional LELVM for the classification.