An efficient and robust sequential algorithm for background estimation in video surveillance

Many computer vision algorithms such as object tracking and event detection assume that a background model of the scene under analysis is known. However, in many practical circumstances it is unavailable and must be estimated from cluttered image sequences. We propose a sequential technique for background estimation in such conditions, with low computational and memory requirements. The first stage is somewhat similar to that of the recently proposed agglomerative clustering background estimation method, where image sequences are analysed on a patch by patch basis. For each patch location a representative set is maintained which contains distinct patches obtained along its temporal line. The novelties lie in iteratively filling in background areas by selecting the most appropriate candidate patches according to the combined frequency responses of extended versions of the candidate patch and its neigh-bourhood. It is assumed that the most appropriate patch results in the smoothest response, indirectly enforcing the spatial continuity of structures within a scene. Experiments on real-life surveillance videos demonstrate the efficacy of the proposed method.