Research on Video Segmentation via Active Learning

We propose a method to video segmentation via active learning. Shot segmentation is an essential first step to video segmentation. The color histogram-based shot boundary detection algorithm is one of the most reliable variants of histogram-based detection algorithms. It is not unreasonable to assume that the color content does not change rapidly within but across shots. Thus, we present a metric based on blocked color histogram (BCH) for inter-frame difference. Our metric is the normalized intersection of BCH between contiguous frames. Hard cuts and gradual shot transitions can be detected as valleys in the time series of the differences between color histograms of contiguous frames or of frames a certain distance apart. We try to estimate the valleys on the frame-to-frame difference curve. Each kind of shot transition (cut or gradual shot transition) has its own characteristic pattern corresponding with valleys. Therefore shot detection can be viewed as pattern recognition. We employ the support vector machine (SVM) via active learning to classify shot boundaries and non-boundaries. Our method is evaluated on the TRECVID benchmarking platform and the experimental results reveal the effectiveness and robustness of the method.