Combination of thermal and color images for accurate foreground / background segmentation in outdoor environment

In the context of outdoor video surveillance, this paper attempts to answer the following question: how to combine LWIR and color information in order to optimize foreground / background segmentation accuracy. Starting from an improved state-of-the-art color-based approach, we integrated thermal information into the algorithm with a pixel-level analytical fusion technique. Considering that very few public thermal / color video databases are available, we built our own acquisition platforms to grab numerous co-registered LWIR / color videos in a variety of outdoor conditions. We manually generated the pixel-based ground-truth for a representative selection of these sequences and performed a quantitative performance analysis. We demonstrated, among others, that the combination of thermal and color information proposed outperformed the use of a single spectral band in all tested visibility conditions.