Robust optical flow estimation in MPEG sequences

Motion information is essential in many computer vision and video analysis tasks. Since MPEG is still one of the most prevalent formats for representing, transferring and storing video data, the analysis of its motion field is important for real time video indexing and segmentation, event analysis and surveillance applications. Our work considers the problem of improving the optical flow field in MPEG sequences. We address the issues of robust, incremental, dense optical flow estimation by combining information from two different velocity fields, the available MPEG motion field and the one inferred by a multiresolution robust regularization technique applied on the DC coefficients. Thus, the regularization technique is based only on information that is directly available in the compressed stream, therefore avoiding time and memory consuming decompression. We extend standard techniques by adding a temporal continuity and an MPEG consistency constraint, both as mathematical constraints in the objective function and as hypothesis tests for the presence of motion discontinuities. Our approach is shown to perform well over a range of different motion scenarios and can serve as a basis for efficient video analysis tasks.