A off-line stroke-based handwritten word segmentation and recognition method for low-quality educational videos

We propose a fast stroke-based method for off-line handwritten word recognition, particularly adapted for use with semantic educational video indexing (blackboard and hand drawn slides taken from low quality videos). We first extract a skeleton from each handwritten word image, then break the skeleton into a sequence of strokes. For each candidate vocabulary word, we use a dynamic programming algorithm on the stroke sequence to incorporate character segmentation and recognition into one procedure, where we find the optimal segmentation and similarity score between word and image simultaneously. Although our method is independent of the implementation of the handwritten character recognition(HCR) module itself, we also propose a stroke-based HCR approach. Stroke matching is based on a set of semantic rules, which are fast and robust to writing styles. We evaluate the overall approach on three different training sets, and the results in this difficult domain are encouraging.