A Tensor Algebraic Approach to Image Synthesis, Analysis and Recognition

We review our multilinear (tensor) algebraic framework for image synthesis, analysis, and recognition. Natural images result from the multifactor interaction between the imaging process, the illumination, and the scene geometry. Numerical multilinear algebra provides a principled approach to disentangling and explicitly representing the essential factors or modes of image ensembles. Our multilinear image modeling technique employs a tensor extension of the conventional matrix singular value decomposition (SVD), known as the N-mode SVD. This leads us to a multilinear generalization of principal components analysis (PCA) and a novel multilinear generalization of independent components analysis (ICA). As example applications, we tackle currently significant problems in computer graphics, computer vision, and pattern recognition. In particular, we address image-based rendering, specifically the multilinear synthesis of images of textured surfaces for varying viewpoint and illumination, as well as the multilinear analysis and recognition of facial images under variable face shape, view, and illumination conditions. These new multilinear (tensor) algebraic methods outperform their conventional linear (matrix) algebraic counterparts.