Detecting events from continuous media by intermodal collaboration and knowledge use

We propose an event network, which is a structured representation oriented for the contents of continuous media, as well as present two methods of detecting events as the first step to construct the network. We deal with sports TV programs, considering American football as a case study. The first method is simple intermodal collaboration: linking between visual and linguistic (closed caption) streams. Using domain knowledge about state transitions of football games, the second method attempts to extract specific visual objects including the information about contents. The experimental results indicate that both methods are effective for event detection