Speaker diarization based on audio-visual integration for smart posterboard

We present a speaker diarization method based on an audio-visual integration approach. We deal with poster conversations which are more challenging than general meetings, because participants are moving freely and the audience utter infrequently. In this case, it is difficult to detect "who spoke when" by only using acoustic information. Therefore we incorporate visual information to improve diarization accuracy. We propose two integration methods: rule-based and stochastic method. Experiments in real poster conversations show that the integration methods significantly outperform the baseline method which uses acoustic information only.