Accurate overlay text extraction for digital video analysis

We present a system to detect and extract overlay text in digital video. To overcome the problems of the previous approaches, we employed a multiple hypothesis filtering approach: the subimages in the region-of-interests (ROI) detected by a localization procedure are decomposed into several hypothetical binary images using color space partitioning; afterwards, the text lines are identified in each binary image using character block grouping and rule-based layout analysis. Moreover, motion verification is used to reduce false alarms. In order to achieve real time speed, the ROI localization procedure is realized using compressed domain features including DCT coefficients and motion vectors in MPEG videos. The proposed method showed impressive results with average recall 96.9% and precision 71.6% when tested in digital news videos. In addition, it demonstrated competitive performance against other systems when tested in NIST-TREC 2002 video data.