Particle swarm optimization based fusion of near infrared and visible images for improved face verification

This paper presents two novel image fusion schemes for combining visible and near infrared face images (NIR), aiming at improving the verification performance. Sub-band decomposition is first performed on the visible and NIR images separately. In both cases, we further employ particle swarm optimization (PSO) to find an optimal strategy for performing fusion of the visible and NIR sub-band coefficients. In the first scheme, PSO is used to calculate the optimum weights of a weighted linear combination of the coefficients. In the second scheme, PSO is used to select an optimal subset of features from visible and near infrared face images. To evaluate and compare the efficacy of the proposed schemes, we have performed extensive verification experiments on the IRVI database. This database was acquired in our laboratory using a new sensor that is capable of acquiring visible and near infrared face images simultaneously thereby avoiding the need for image calibration. The experiments show the strong superiority of our first scheme compared to NIR and score fusion performance, which already showed a good stability to illumination variations.