Ensemble Manifold Structured Low Rank Approximation for Data Representation

Graph regularized techniques have been extensively exploited in unsupervised learning. However, there exist no principled ways to select reasonable graphs and their associated hyper parameters, particularly in multiple heterogeneous data sources. Often, the graph selection process requires rather time-consuming cross-validation and discrete grid search that are not scalable to a large number of candidate graph sources. To address this issue, we propose a new formulation by integrating Ensemble Manifold structure into Low Rank approximation (EMLR). The central idea is to maximally approximate the intrinsic geometric structure by searching the optimal linear combination space of multiple different graphs. Specifically, efficient projection onto the probabilistic simplex is utilized to optimize the graph weights, resulting in the sparsity pattern of coefficients. This attractive property of sparsity can be properly interpreted as a criterion for selection of graphs, i.e., identifying most discriminative graphs and removing noisy or irrelevant graphs under the low rank decomposition model. Therefore, the compact output representation and linear combination coefficients of multiple different graphs can be simultaneously achieved by a unified objective. Exhaustive experimental results corroborate the effectiveness of our new model.