A Bayesian 3D People Tracker using Multiple Cameras and a Microphone Array

In this paper, we consider the problem of tracking multiple people in a 3D world domain using a microphone array and multiple cameras. The data fusion is done using a particle filter. To support 3D tracking, we propose a new video data likelihood model using a camera calibration matrix that can be used for a moving camera without continuous camera calibration. Then we apply an independent partition particle filter for multiple objects in order to generate particles efficiently. To detect the current speaker, we use a simple cost function using the generated particles. Finally we implement this tracking algorithm as a real-time system.