Dynamic mutual calibration and view planning for cooperative mobile robots with panoramic virtual stereo vision

This paper presents a panoramic virtual stereo vision approach to the problem of detecting and localizing multiple moving objects (e.g., humans) in an indoor scene. Two panoramic cameras, residing on different mobile platforms, compose a virtual stereo sensor with a flexible baseline. A novel “mutual calibration” algorithm is proposed, where panoramic cameras on two cooperative moving platforms are dynamically calibrated by looking at each other. A detailed numerical analysis of the error characteristics of the panoramic virtual stereo vision (mutual calibration error, stereo matching error, and triangulation error) is given to derive rules for optimal view planning. Experimental results are discussed for detecting and localizing multiple humans in motion using two cooperative robot platforms.