Sensor fusion for model based 3D tracking

In this paper, we present a new approach for fusion of different measurements and sensors for 3D model based tracking. The underlying model of the tracked body is defined geometrically with generalized cylinders, which can hierarchically be connected by different kinds of joints. This results in an articulated body model with constrained kinematic degrees of freedom. The fusion approach incorporates this model knowledge together with the measurements, and tracks the target body iteratively with an extended iterative closest point approach. The resulting tracking system named VooDoo is used to track humans in a human-robot interaction (HRI) context. We only rely on sensors on board the robot, i.e. a color camera, a 3D time-of-flight camera and a laser range finder. The system runs in realtime (~ 20 Hz) and is able to robustly track a human in the vicinity of the robot. The pose and trajectory of the human interaction partner can then be used for haptic interaction like hand-overs, and for activity and gesture recognition