3D scene flow estimation with a rigid motion prior

We present an approach to 3D scene flow estimation, which exploits that in realistic scenarios image motion is frequently dominated by observer motion and independent, but rigid object motion. We cast the dense estimation of both scene structure and 3D motion from sequences of two or more views as a single energy minimization problem. We show that agnostic smoothness priors, such as the popular total variation, are biased against motion discontinuities in viewing direction. Instead, we propose to regularize by encouraging local rigidity of the 3D scene. We derive a local rigidity constraint of the 3D scene flow and define a smoothness term that penalizes deviations from that constraint, thus favoring solutions that consist largely of rigidly moving parts. Our experiments show that the new rigid motion prior reduces the 3D flow error by 42% compared to standard TV regularization with the same data term.