A new framework for video cut and paste

In this paper, we describe a novel framework to cut and paste objects among different video shots. First, based on geometric analysis of the camera motion of a shot, we classify the shot into either simple camera motion shot, e.g. panning, tracking and zooming, or complex camera motion shot, e.g. hand-held shots. Next, for the simple camera motion shots, we temporally align them captured by cameras undergoing the same motions based on the quantization in the geometric analysis. In the case of complex camera motion, we recover the camera poses for each frame of both the source and target shots and, thus, for each frame in the target shot, we find the corresponding source frame with the closest viewing direction. Then, the foreground objects in the source shots are automatically cut by combining the merits of motion layer segmentation and alpha matting techniques. Finally, the extracted foreground mattes can be directly blended into the corresponding target frames for simple motion shots. For complex shots, combining the estimated rough depths of the foreground objects, foreground layers are rendered and blended into the target frames. Results on various content types, e.g. home videos and feature films, and different camera motions are reported for validating the proposed framework