Object-Based Visual 3D Tracking of Articulated Objects via Kinematic Sets

A theoretical framework based on robotics techniques is introduced for visual tracking of parametric non-rigid multi-body objects. It is based on an a-priori model of the object including a general mechanical link description. The objective equation is defined in the object-based coordinate system and non-linear minimization relates to the movement of the object and not the camera. This results in simultaneously estimating all degrees of freedom between the object´s last known position relative to its previous position as well as internal articulated parameters. A new kinematic-set formulation takes into account that articulated degrees of freedom are directly observable from the camera and therefore their estimation does not need to pass via a kinematic-chain back to the root. By doing this the tracking techniques are efficient and precise leading to real-time performance and accurate measurements. The system is locally based upon an accurate modeling of a distance criteria. A general method is derived for defining any type of mechanical link and experimental results show prismatic, rotational and helicoidal type links. A statistical M-estimation technique is applied to improve robustness. A monocular camera system was used as a real-time sensor to verify the theory.