Real-Time Fusion of Multimodal Tracking Data and Generalization of Motion Patterns for Trajectory Prediction

A sensor-based model of a service robot´s environment is a prerequisite for interaction. Such a model should contain the positions of the robot´s interaction partners. Many reasonable applications require this knowledge in realtime. It could for example be used to realize efficient path planning for delivery tasks. Additionally to the actual positions of the partners it is important for the service robot to predict their possible future positions. In this paper we propose an extensible framework that combines different sensor modalities in a general real-time tracking system. Exemplarily, a tracking system is implemented that fuses tracking algorithms in laser range scans as well as in camera images by a particle filter. Furthermore, human trajectories are predicted by deducing them from learned motion patterns. The observed trajectories are generalized to trajectory patterns by a novel method which uses self organizing maps. Those patterns are used to predict trajectories of the currently observed persons. Practical experiments show that multimodality increases the system´s robustness to incorrect measurements of single sensors. It is also demonstrated that a self organizing map is suitable for learning and generalizing trajectories. Convenient predictions of future trajectories are presented which are deduced from these generalizations.