Temporally consistent video matting based on bilayer segmentation

In this paper, a quasi-automatic video matting approach which can preserve the temporal consistency of the alpha mattes is presented. “Quasi-automatic” means that it only needs a few user interactions on the first frame. A new algorithm which incorporates the Bayesian Estimation, Weighted Kernel Density Estimation (WKDE) and graph cut is presented to automatically and accurately segment each frame into bilayer. And then a GMM (Gaussian Mixture Model) based algorithm is used to transform the foreground layer to the trimap. At last, a 3D closed-form method is used to achieve the temporally consistent video matting. In order to give the user real time feedbacks when the user adds more scribbles to edit the matting result, a smallest-independent-region based re-matting algorithm is proposed. We test our algorithm with variety of video sequences, and the results demonstrate the effectiveness of our approach.