Object tracking by comparing CG images with multiple viewpoint images

A method for 3D (three-dimensional) object motion tracking without feature extraction is necessary for monitoring human action in a usual civil-life scene. In order to realize a fast and robust object tracking method, we propose a model-based method using intensity images taken with a multiple viewpoint camera connected to a PC cluster system. At first, the whole 3D shape and reflectance of the object are prepared using several rangefinders. Each rangefinder is constructed with a camera, projector and PC (personal computer), and all PCs are connected with each other via a network. For tracking the object, several CG images with varied object pose and position are generated in each PC using the object model, and then compared to the input intensity image in parallel. The result of the comparison is transferred to a master PC, and the pose and position of the object are estimated by minimizing the residual of the CG and input images. We made a special CG generator, which is a precise simulator of the real camera to generate a CG image identical to the input image. We confirmed the ability of our method and achieved a 3DOFs real-time object tracking system in 4.5 frames per second speed.