Fast, robust and occlusion resilient motion based video segmentation

Segmentation using tracked points is an active research area with various applications in video processing. Many algorithms have been proposed and some have shown promising results. Nonetheless, most methods have overlooked the problem of object occlusion. For instance, two objects that move similarly before a partial occlusion, may be merged even if their motions differ afterwards. The occluded parts provide a spurious link that can lead to undersegmentation. In this paper, following the framework of previous work where a video is divided in subsequences, we focus on the negative effect of occlusions and propose a method that resolves such cases. We improve the quality and robustness of segmentation by rejecting noisy trajectories using the sparsity of reprojection error. Finally, we propose a new metric to measure the motion dissimilarity between segments. The experimental evaluation shows that our algorithm achieves competitive results while being significantly faster than other methods.