Combination of colour and thermal sensors for enhanced object detection

In uncontrolled environments, with dynamic background and lighting changes, performing efficient and real-time foreground - background segmentation is very challenging. This work is based on the hypothesis that the combination of long wave infrared (LWIR) (8-12 mum) and colour cameras can significantly improve the robustness of moving objects extraction. Pros and cons of colour and thermal imagers in outdoor video monitoring applications are discussed. In order to fuse information from both sensors, we favoured an approach based on "analytical fusion" rather than "representative fusion". Starting from a state-of-the-art algorithm for moving objects extraction in colour video (non-parametric codebook model), we first adapted the method for processing of "red-green-blue-thermal" video format. A preliminary objective performance evaluation of detection accuracy is presented. Original image sequences grabbed with co-aligned thermal and visible fields of view was used. Finally, some improvements to the original codebook model are proposed.