Locality preserving KSVD for nonlinear manifold learning

Discovering the intrinsic low-dimensional structure from high-dimensional observation space (e.g., images, videos), in many cases, is critical to successful recognition. However, many existing nonlinear manifold learning (NML) algorithms have quadratic or cubic complexity in the number of data, which makes these algorithms computationally exorbitant in processing real-world large-scale datasets. Randomly selecting a subset of data points is very likely to place NML algorithms at the risk of local optima, leading to poor performance. This paper proposes a novel algorithm called Locality Preserving KSVD (LP-KSVD), which can effectively learn a small number of dictionary atoms as locality-preserving landmark points on the nonlinear manifold. Based on the atoms, the computational complexity of NML algorithms can be greatly reduced while the low-dimensional embedding quality is improved. Experimental results show that LP-KSVD successfully preserves the geometric structure of various nonlinear manifolds and it outperforms state-of-the-art dictionary learning algorithms (MOD, K-SVD and LLC) in our preliminary study on face recognition.