An accurate 3D feature tracking system with wide-baseline stereo smart cameras

A typical video surveillance system consists of at least one camera, controlled by an operator. To decrease the human error rate and to generally lessen the burden of operators, many object tracking systems have been implemented, most of which work in 2D image space. If used centralized, this is a very expensive task. Furthermore, if several views are to be fused, large inaccuracies arise due to ground plane assumptions, for instance. Lastly, in outdoor setups, quite often there is a need for slower channels like Wireless LAN which cannot cope with the full resolution data stream. We provide a smart camera system which performs the intensive tasks like background estimation or feature extraction. A central unit only has to process the received data in feature space, increasing scalability. Additionally, the object tracking problem is converted to an accurate 3D feature tracking, avoiding difficulties such as proper object segmentation and adding increased trajectory accuracy. The feature regions are computed within the smart camera. A wide-baseline feature matching approach has been employed to allow more freedom in the placement of the single smart cameras.