Video text detection with text edges and convolutional neural network

Text and captions in videos provide useful information for content analysis and understanding. In this paper, we present an approach to detecting video text in a coarse-to-fine strategy. In the coarse phase we propose an efficient method to detect multi-scale candidate text regions with high recall. Then the candidate text regions are segmented and sent to the fine phase where a convolutional neural network(CNN) is applied to generate a confidence map for each candidate text region. Finally, the candidate text regions are further refined and partitioned into text lines by projection analysis. The CNN classifier in the fine phase enables feature sharing and robustly identifies text regions. The coarse phase sharply reduce the number of windows needed to be scanned by the CNN. The combination endows the proposed method with both efficiency and robustness when detecting video text. It was verified by experiment results on two publicly testing datasets and a dataset created by us.