Towards ultimate motion estimation: combining highest accuracy with real-time performance

Although variational methods are among the most accurate techniques for estimating the optical flow, they have not yet entered the field of real-time vision. Main reason is the great popularity of standard numerical schemes that are easy to implement, however, at the expense of being too slow for real-time performance. In our paper we address this problem in two ways: (i) we present an improved version of the highly accurate technique of Brox et al. (2004). Thereby we show that a separate robustification of the constancy assumptions is very useful, in particular if the I-norm is used as penalizer. As a result, a method is obtained that yields the lowest angular errors in the literature, (ii) We develop an efficient numerical scheme for the proposed approach that allows real-time performance for sequences of size 160 × 720. To this end, we combine two hierarchical strategies: a coarse-to-fine warping strategy as implementation of a fixed point iteration for a non-convex optimisation problem and a nonlinear full multigrid method - a so called full approximation scheme (FAS) - for solving the highly nonlinear equation systems at each warping level. In the experimental section the advantage of the proposed approach becomes obvious: Outperforming standard numerical schemes by two orders of magnitude frame rates of six high quality flow fields per second are obtained on a 3.06 GHz Pentium4 PC.