Motion segmentation via overlapping temporal windows

In this paper we present a novel approach to motion segmentation. Initially, the video sequence is divided in overlapping temporal windows. Our algorithm performs over-segmentation on each window separately. Concretely, quadruples of trajectories are used as motion subspaces and the Ordered Residual Kernel is employed as an affinity metric between trajectories. The corresponding graph of the computed affinity matrix is partitioned via a random walk algorithm. A motion dissimilarity score is proposed to correlate the computed segments as well as a merging mechanism that fuses the individual segmentation results of successive windows. Experiments on the Berkeley motion segmentation dataset demonstrate the scalability and accuracy of our method compared to the existing approaches.