Frame rate up-conversion using global and local higher-order motion

A region-based frame rate up-conversion (FRUC) algorithm based on higher-order global and local motion is proposed in this paper. First, perspective global motion parameters are estimated so that backgrounds of neighboring frames can be aligned. Then, the foreground is separated from the background using structural similarity, and iteratively decomposed into regions with homogeneous motion that fit a local affine or perspective model. Also, regions entering or leaving the scene are identified. Finally, an intermediate frame is synthesized by warping the background and foreground regions towards the desired frame location using their associated motion parameters. Experimental results show that the proposed algorithm outperforms several recent FRUC methods in both PSNR and SSIM, and produces more aesthetically pleasing frames, free of blocking artifacts.