Fusion of thermal imaging and CCD camera-based data for stereovision visual telepresence

The aim of this paper is to describe the fusion of data acquired from a CCD camera and a thermal imager. The fusion is realized by means of spatial data from a TOF camera to ensure “natural” representation of a robot´s environment; thus, the thermal and color-related data are comprised in one stereo image presented to a binocular, head-mounted display. The data acquisition is performed using a sensor head, which is placed on an Orpheus-X3 robot; both the head and the robot were developed by our working group. The head contains five matrix sensors: a pair of CCD cameras, a pair of thermal imagers, and one TOF camera. After the geometrical calibration of each sensor, the positions of the sensors in 6DOFs are computed. The corresponding data from the CCD camera and the thermal imager are subsequently determined via homogeneous and perspective transformations. The result consists in an image containing aligned data from the CCD camera and the thermal imager for each eye.