Joint trilateral filtering for multiframe optical flow

Since two years there is a recent trend in optical flow estimation to improve the results of state-of-the-art variational methods by applying additional filtering steps such as median filters, bilateral filters, and non-local techniques. So far, however, the application of such filters has been restricted to two-frame optical flow methods. In this paper, we go beyond this two-frame case and investigate the usefulness of such filtering steps for multi-frame optical flow estimation. Thereby we consider both the application to single flow fields as well as the filtering of the entire spatio-temporal flow volume. In this context, we propose the use of a joint trilateral filter that processes all flow fields simultaneously while imposing consistency of joint flow structures at the same time. Evaluations on the Middlebury benchmark clearly demonstrate the success of our filtering strategy. Achieving rank 3, our method yields state-of-the art results and significantly outperforms the baseline method providing considerably sharper results.