Generating metadata from acoustic and speech data in live broadcasting

This paper describes a method to generate metadata for TV programs in real-time by utilizing acoustic and speech data in live broadcasting. Various styles of watching TV programs can be provided by using metadata related to the content of the program. The acoustic data to be processed in our case is crowd noise in a football (soccer) stadium, and the speech data is an announcer´s voice. The crowd noise is closely related to not only spectator emotions but also their attention and expectations. In other words, a part in which the crowd noise rises corresponds to an important event in the game. Because the crowd noise conveys no further information about what happened in the scene, the announcer´s voice, after speech-to-text conversion, is processed to extract further meaning. By combining these two processes of identifying and extracting, content-based segment metadata is generated automatically. This method was applied to generating metadata for six professional football games, by which its effectiveness was verified.