Incremental Unsupervised Three-Dimensional Vehicle Model Learning From Video

In this paper, we present a new generic model-based approach for building 3-D models of vehicles from color video from a single uncalibrated traffic-surveillance camera. We propose a novel directional template method that uses trigonometric relations of the 2-D features and geometric relations of a single 3-D generic vehicle model to map 2-D features to 3-D in the face of projection and foreshortening effects. We use novel hierarchical structural similarity measures to evaluate these single-frame-based 3-D estimates with respect to the generic vehicle model. Using these similarities, we adopt a weighted clustering technique to build a 3-D model of the vehicle for the current frame. The 3-D features are then adaptively clustered again over the frame sequence to generate an incremental 3-D model of the vehicle. Results are shown for several simulated and real traffic videos in an uncontrolled setup. Finally, the results are evaluated by the same structural performance measure, underscoring the usefulness of incremental learning. The performance of the proposed method for several types of vehicles in two considerably different traffic spots is very promising to encourage its applicability in 3-D reconstruction of other rigid objects in video.