Evaluation of video segmentation methods for surveillance applications

Current research on performance evaluation of video segmentation methods is primarily focussed on the development of objective figures of merit. There is no standardised methodology for subjective evaluations of segmentation performance and these are currently perceived as too onerous. Using an experimental design and data analysis method derived from current practice in experimental psychology, we have explored a performance evaluation procedure that is largely based on subjective assessments. We report on statistically significant differences in perceived performance between three multi-object video segmentation and tracking methods developed for surveillance applications. The assessments were performed by a group each of experts and novices on a wide range of video content.