Dynamic background substraction for object extraction using virtual reality based prediction

This paper presents a new approach to background substraction algorithms to extract video objects from a sequence. Rather than working with a fixed, flat background, the system relies on a virtual 3D model of the background that is automatically created and updated using a sequence of images of the environment. Each time an image is captured, the position of the camera is estimated and the corresponding view of the background can be rendered. The substraction between the frame and the view provides video objects not present in the background. In order to estimate the position of the camera to create the background model and render a background view, artificial landmarks of known size are distributed in the environment. The system works correctly in real environments, over 20 frames per second. It recovers from illumination changes and automatic white balance (AWB) thanks to our background updating algorithm