Exploiting Temporal Geometry for Moving Camera Background Subtraction

In this paper, we introduce a new method for background subtraction for freely moving cameras and arbitrary scene geometry. Instead of relying on frame-to-frame estimation, we simultaneously estimate the epipolar geometries induced by a moving camera in a temporally consistent manner for a number of frames using the temporal fundamental matrix (TFM). The TFM is robustly estimated from the track lets generated by dense point tracking and used to compute the probability of each track let belonging to the background. In order to ensure the color, spatial, and temporal consistencies of track let labeling, we minimize spatiotemporal labeling cost in locality of track lets. Extensive experiments with challenging videos show that the proposed method is comparable and, in most cases, outperforms the state-of-the-art.