3D augmented Markov random field for object recognition

In this paper, we propose the use of 3D information to augment the Markov random field (MRF) model for object recognition. Conventional MRF for image-based object recognition usually uses appearance and 2D location as features in the model. We estimate rough 3D information from stereo image pairs, and incorporate this information into node and edge potential models in the conventional MRF. Introducing 3D information into the node potential allows to leverage the distribution statistics of 3D location for different classes. We solve the object recognition problem by finding the globally optimal class assignment that minimizes an energy function defined in the augmented MRF. We show that the introduction of 3D distance in the edge potential can help distinguish “true” neighbors from “fake” neighbors in 2D. We demonstrate improved recognition results by using the proposed technique.