Graph regularized multiset canonical correlations with applications to joint feature extraction

Multiset canonical correlation analysis (MCCA) is a powerful technique for analyzing linear correlations among multiple representation data. However, it usually fails to discover the intrinsic geometrical and discriminating structure of multiple data spaces in real-world applications. In this paper, we thus propose a novel algorithm, called graph regularized multiset canonical correlations (GrMCCs), which explicitly considers both discriminative and intrinsic geometrical structure in multiple representation data. GrMCC not only maximizes between-set cumulative correlations, but also minimizes local intraclass scatter and simultaneously maximizes local interclass separability by using the nearest neighbor graphs on within-set data. Thus, it can leverage the power of both MCCA and discriminative graph Laplacian regularization. Extensive experimental results on the AR, CMU PIE, Yale-B, AT&T, and ETH-80 datasets show that GrMCC has more discriminating power and can provide encouraging recognition results in contrast with the state-of-the-art algorithms.