Extracting textual inserts from digital videos

Textual inserts and closed captures superimposed on digital videos often contain important and exclusive information about the video contents which cannot be found in other information channels. Therefore, it is very helpful to extract this information automatically and add it to a video index as generated by video archiving and retrieval systems like e.g. ADViSOR, AVAnTA, DiVA or Informedia. Owing to the fact that common OCR systems are restricted to binary images, the video frames have to be preprocessed in order to extract the textual inserts from the image in the background. In this paper we present our approach to the segmentation of textual inserts from digital videos or images, which consists of a region-growing method for color segmentation and a method of separating text regions from background based on character size and alignment constraints. A new method on segmentation refinement taking into account the results of the classification step leads to a significant enhancement of quality of the resulting binary images. The main difficulties in extracting textual inserts from video are caused by the low resolution and quality of digital video material, the high amount of image data, the very complex structured and textured background, and the unknown color size, and position of the text to be extracted from the image.