Speaker detection and tracking at mobile robot platform

In this paper, we present a novel method to detect and track multiple people for an audio-visual speaker localization system using an uncalibrated camera and an 8 microphone array. We propose an object oriented speaker detection and tracking algorithm that uses a combination of skin color and human upper body part appearance information. Candidate regions of the input image are extracted from the color transform that is modeled by a 2D Gaussian function in a normalized RGB color space. We classify the humans and other objects which have a similar skin color region from the geometric structure and Hausdorff distance from the human shape. We decide the location of the speaker from audio and visual information and the robot tracks the speaker using CAMShift. The experimental results show that the proposed algorithm initializes and tracks the speaker automatically in various environments such as an image with skin color noise and complex background.