Estimating Mixing Factors Simultaneously in Multilinear Tensor Decomposition for Robust Face Recognition and Synthesis

Facial images change appearance due to multiple factors such as poses, lighting variations, facial expressions, etc. Tensor approach, an extension of conventional matrix, is appropriate to analyze facial factors since tensors make it possible to construct multilinear models using a multiple factor structure. We use higher-order tensors to model multiple factors of facial variations, but this provides some difficulty in use. First, it is difficult to decompose the multiple factors of a test image, especially when the factor parameters are unknown or are not in the training set. Second, for face recognition and face synthesis tasks, it is also difficult to construct reliable multilinear models which have more than two factors. In this paper, we propose a novel tensor factorization method to decompose mixing factors for unseen test images. We set up tensor factorization problem as a least squares problems with a quadratic equality constraint, and solve it using numerical optimization techniques. The novelty in our approach compared to previous work is that our tensor factorization method does not require any knowledge or assumption of test images; we can attain parameters of facial factors by our tensor factorization method. We have conducted several experiments to show the versatility of the method for both face recognition and synthesis tasks. Thus, we demonstrate the proposed method produces reliable results for trilinear models as well as bilinear models.