Adaptive frame selection for improved face recognition in low-resolution videos

Performing face detection and identification in low-resolution videos (e.g., surveillance videos) is a challenging task. The task entails extracting an unknown face image from the video and comparing it against identities in the gallery database. To facilitate biometric recognition in such videos, fusion techniques may be used to consolidate the facial information of an individual, available across successive low-resolution frames. For example, super-resolution schemes can be used to improve the spatial resolution of facial objects contained in these videos (image-level fusion). However, the output of the super-resolution routine can be significantly affected by large changes in facial pose in the constituent frames. To mitigate this concern, an adaptive frame selection technique is developed in this work. The proposed technique automatically disregards frames that can cause severe artifacts in the super-resolved output, by examining the optical flow matrices pertaining to successive frames. Experimental results demonstrate an improvement in the identification performance when the proposed technique is used to automatically select the input frames necessary for super-resolution. In addition, improvements in output image quality and computation time are observed. The paper also compares image-level fusion against score-level fusion where the low-resolution frames are first spatially interpolated and the simple sum rule is used to consolidate the match scores corresponding to the interpolated frames. On comparing the two fusion methods, it is observed that score-level fusion outperforms image-level fusion.