Background recognition in dynamic scenes with motion constraints

Consider a monocular image sequence which contains independently moving objects and assume it is already segmented. In order to get a realistic 3D reconstruction of such a scene, we have to solve the relative scale ambiguity between the reconstructions of different moving objects. Recently, we demonstrated the usefulness of the so-called ´non-accidentalness and independence constraints´ to disambiguate the mentioned unknown relative scale. However, this technique requires that the video segment which corresponds to the background is known beforehand. In this paper, we analyze the background detection problem in the vein of the aforementioned constraints and show that the background is not just another moving object but the one which results in the simplest overall scene interpretation.