Person tracking in enhanced cognitive care: A particle filtering approach

We propose an audiovisual system for detecting the active speaker in cluttered and reverberant environments where more than one person speaks and moves. The feasibility of the systems is examined in the context of smart-houses supporting elderly living alone. Rather than using only audio, the system utilizes audiovisual information from a minimal setup comprising of three microphones and one camera feeding separate audio and visual tracking modules. The audio module operates using a Particle Filter (PF) in order to provide accurate acoustic source location under reverberant conditions. The visual module combines with a second PF video cues generated by colour and face measurements. The final decision is performed through the employment of a fusion mechanism that combines the estimate of each modality according to the current observations. Results indicate that the performance of the proposed multi-modal tracking can potentially enable services for the elderly in their domestic environment if those require knowledge of speaker.