Coping with 3D artifacts in video sequences

Several video processing techniques with applications in video compression, shot-cut detection, mosaicing and so on are based around the mapping of the positions of feature points from video frame to video frame. This in turn is often based on the assumption that for the scene under observation, a two dimensional approximation of the structure is adequate - both in terms of motion estimation and in the subsequent use of the feature correspondences obtained from the motion estimation process. This assumption is in general false. We propose a method for motion estimation based on an active mesh. In doing so we attempt to model the motion estimation not as an unconstrained motion of feature points moving independently, but as the estimation of a set of planar patches each undergoing a 3D perspective motion. In doing so we note that our method allows us to fit to 3D metrics with a high degree of accuracy