Video-to-video face authentication system robust to pose variations

High-quality still-to-still (image-to-image) face authentication has shown success under controlled conditions in many safety applications. However, video-to-video face authentication is still challenging due to appearance variations caused by pose changes. In this paper, we propose a video-to-video face authentication system that is robust to pose variations by making use of synthesized frontal face appearance that contains both texture and shape information. To obtain the appearance, we first reconstruct 3D face shape from face feature points detected from the video using active shape model (ASM). Conventional ASM algorithms cannot handle large pose variations and fast head movement exhibited in video sequences. To address these problems, we present a novel prediction-assisted approach that is capable of providing an accurate shape initiation as well as automatically switching on multi-view models for ASM. Then we can generate frontal shape mesh from the reconstructed 3D face shape. Based on the mesh, we synthesize frontal face appearance with the ASM-detected faces in video. For authentication, in order to match the synthesized appearances of enrollment and probe, we propose a 2-directional 2-dimensional client specific fisher’s linear discriminant algorithm. The proposed algorithm is a variant of fisher’s linear discriminant (FLD) and directly computes eigenvectors of image scatter matrices in row and column direction without matrix-to-vector conversion. In experiments, our authentication system is compared with the other state-of-art approaches on public face database and our face database. The results show that our system demonstrates a higher authentication accuracy and pose-robust performance.