Fusion of visible and synthesised near infrared information for face authentication

Changes in illumination conditions can cause drastic variations in face appearance and affect the performance of a face authentication system. Near infrared (NIR) face imaging systems have been proposed as a promising way towards illumination invariant face verification. We show that when NIR face images cannot be observed, learning the relationship between NIR information and the corresponding visible images can provide useful complementary information about visible light image data. In particular, we use Canonical Correlation Analysis (CCA) to synthesise the NIR eigenfaces from their corresponding visible ones. In this paper, the verification performance of a CCA-based synthesising algorithm is developed first. Although, synthesised NIR images do not perform as well as the real NIR, it is shown that by fusing the visible and synthesised near infrared information at the score level, the performance of the authentication system considerably improves.