Back to the future: Robust foreground extraction with reversed-time background modeling

ldquoGhostsrdquo arise in traditional background subtraction when an object starts to move, causing the exposed background to be labelled as a ghost foreground. With background model updates, the ghost may disappear after some time, but removing ghosts immediately is crucial for identifying starts, and also for improving downstream tasks like tracking, object recognition and activity analysis. Here we propose a lagged background subtraction process, where the background model is computed in reverse after a lag of k frames, resulting in a ldquoreversed-timerdquo background model. We present an algorithm that handles disparities between the forward and backward foreground blobs, and show how ghosts can be reliably eliminated, potentially permitting single-frame latency in reliably detecting starts and identifying stops. We present theoretical results that the bidirectional model results in lower false positives (such as ghosts) compared to either directional model, and develop an algorithm for identifying ghosts.