Robust face recognition with occlusions in both reference and query images

Face recognition (FR) systems in real environment need to deal with uncontrolled variations in face images such as occlusions and disguise. Most of the current FR algorithms do not consider the fact that occlusions may exist in both reference and query images. In this paper, we summarise three occlusion cases that a realistic FR system should take account of. We present a novel non-parametric classification method to handle the occlusion related problems. Our method represents a face image as a sub-patch sequence which maintains the inherent structure information of the face. Matching is based on the Image-to-Class distance from a query sequence to all reference sequences of an enrolled class. Experimental results on public databases verify the effectiveness and robustness of the proposed method.