Multi-spectral face recognition: Identification of people in difficult environments

In this paper we study the problems of intra-spectral and cross-spectral face recognition (FR) in homogeneous and heterogeneous environments. Specifically we investigate the advantages and limitations of matching (i) short wave infrared (SWIR) face images to visible images under controlled or uncontrolled conditions, (ii) mid-wave infrared (MWIR) to MWIR or visible images under controlled conditions, and (iii) intra-distance near infrared (NIR) to NIR images and cross-distance, cross-spectral NIR to visible images. All NIR images were captured night-time, outdoors and at mid-ranges (from 30 up to 120 meters). We utilized both commercial and academic face matchers and performed a set of experiments indicating that our cross-photometric score level fusion rule can be utilized to improve SWIR cross-spectral matching performance across all FR scenarios investigated. We also show that intra-spectral matching results, using either MWIR or NIR images, are comparable to the baseline results, i.e., when comparing visible to visible face images. Our experiments also indicate that the level of improvement in recognition performance is scenario dependent. Experiments also show that cross-spectral matching (the heterogeneous problem, where gallery and probe sets have face images acquired in different spectral bands) is a very challenging problem and it requires further investigation to address real-world law enforcement or military situations.