Fusion of Multispectral Video Sequences Based on the Trajectories of Moving Objects

The fusion of images coming from multiple type of sensors (particularly infrared sensor and color sensor) has not been widely explored in the literature. Following the drop of the prices of infrared camera, this situation is about to change. In this work, a first step to combine these two types of images is presented. The main goal of this project is to find the transformation matrix between the two cameras (one infrared, the other color). To do so, the trajectories of the moving objects are used because they are independent of the type of the sensors. These trajectories are matched using a RANSAC-based algorithm. The performance criteria used in this algorithm is based on the number of pixels in intersection after overlapping the two images (one image is from the color camera and the other is obtained by the tested transformation matrix and the infrared image). The results obtained with our proposed method are promising as they are more and more accurate as new trajectory points are added.