Generating hypermedia documents from transcriptions of television programs using parallel text alignment

This paper presents a method of automatically creating hypermedia documents from conventional transcriptions of television programs. Using parallel text alignment techniques, the temporal information derived from the closed caption signal is exploited to convert the transcription into a synchronized text stream. Given this text stream, we can create links between the transcription and the image and audio media streams. We describe a two-pass method for aligning parallel texts that first uses dynamic programming techniques to maximize the number of corresponding words (by minimizing the word edit distance). The second stage converts the word alignment into a sentence alignment, taking into account the cases of sentence split and merge. We present results of text alignment on a database of 610 programs (including three television news programs over a one-year period) for which we have closed caption, transcript, audio and image streams. The techniques presented can produce high quality hypermedia documents of video programs with little or no additional manual effort.