Combining low-level segmentation with relational classification

A novel approach is presented that classifies multiple independently moving objects by taking into account existing object relations, closing the loop to low-level scene segmentation. The method partitions a stereo image sequence into its most prominent moving groups with similar 3-dimensional (3D) motion. Object motion is estimated using the expectation-maximization (EM) algorithm. The EM formulation is used to account for the unknown associations between objects and observations. In a segregation step, each image point is assigned to the object hypothesis with maximum a posteriori (MAP) association probability. This segmentation is fed into a multiple object classification scheme based on Markov logic which integrates relational scene knowledge. Class probabilities for the individual object hypotheses are then used within the association process for track enhancement.