Robust video text segmentation and recognition with multiple hypotheses

A method for segmenting and recognizing text embedded in video and images is proposed. Multiple segmentation of the same text region is performed, thus producing multiple hypotheses of binary text images. The segmentation algorithm is stated as a statistical labeling and is based on a Markov random field (MRF) model of the label map. Background regions in each hypothesis are then removed by performing a connected component analysis and by enforcing a more stringent constraint (called GCC - grayscale consistency constraint) on the text characters´ grayscale values using a robust 1D-median operator. Each text image hypothesis is then processed by optical character recognition (OCR) software. The final result is then selected from the set of output strings. Results show that both the use of multiple hypotheses and the GCC significantly improve the results.