Multiple Object Detection under the Constraint of Spatiotemporal Consistency

We propose a method for partitioning a stereo image sequence of a dynamic 3-dimensional (3D) scene into its most prominent moving groups with similar 3D motion. For this purpose we assign each image point one of a finite number of motion profiles. Each profile describes one dominant 3D motion in the imaged scene, i.e. translational and rotational 3D motion. Image segmentation is performed by assignment of the most probable motion profile to each image point. While segmentation approaches known in literature often lack spatial coherence of object points, the algorithm presented in this paper further accounts for the intuitive notion that points belonging to the same motion also tend to be spatially clustered in the image. We construct a graph encoding the spatiotemporal image point affinity in a neighborhood around each point. The spatial coherence of neighboring points is modeled by a Markov random field (MRF) and is optimized with recently proposed graph-cut methods. The motion profiles of the elaborated motion models are iteratively refined by an object tracking procedure. The continuous interaction of object tracking and image segmentation provides an object detection process whose performance improves over time. Another advantage of the proposed method is that it avoids heuristic presumptions, e.g. on the shape of objects in the scene. This makes the algorithm generally applicable to a wide range of outdoor traffic scenarios and also highly adaptable to general object detection methods. Results are presented on real and synthetic image sequences