Region of interest extraction using colour based methods on the CUAVE database

Region of interest (ROI) extraction is an important step in deriving visual features for an audio-visual speech recognition system. Colour based segmentation offers the potential of computationally inexpensive algorithms for ROI selection. This paper presents a comparative study of two colour based techniques, one using hue and accumulated difference, the other chrominance. Results are presented for the CUAVE database. The two methods achieved 69% and 72% correct ROI extraction. The experiment prompted investigation of a new method using a chrominance based accumulated difference image. The new method achieved 79% correct ROI identification. The overall results suggest that a dual approach using chrominance to locate the mouth region and only employing an accumulated difference image when significant motion is not present would offer good robustness with lower computational cost.