Semantic segmentation of TV programs using corner-subtitles

In this paper, we propose a new semantic segmentation method for TV programs based on detection and classification of corner-subtitles. Corner-subtitle refers to subtitles that are found on the corner of the screen. They are widely used in Japanese TV programs to present information related to a scene. Corner-subtitles can be used as effective indexes indicating the structure of the program because they stay on the screen and switch at semantic scene changes. However, since they are often shown in small size and tint color to avoid obstructing the image, it is difficult to detect them stably. Our method replaces detection of complicated subtitles with detection of straight lines in a spatio-temporal image to enable stable detection, and programs are chaptered according to the subtitles. Experiments show the effectiveness of chaptering of individual sports, variety, and informational programs which are difficult to index by conventional image or acoustic features.