Hierarchical road understanding for intelligent vehicles based on sensor fusion

Comprehensive situational awareness is paramount to the effectiveness of proprietary navigational and higher-level functions of the intelligent vehicles. In this paper, we address a hierarchical road understanding system for intelligent vehicles with respect to the road topography and the existence of objects based on sensor fusion. The proposed system consists of three modules that run in parallel. Module one classifies the road environment into four categories, i.e. the reachable region, the drivable region, the obstacle region and the unknown region. In module two, an efficient graph-based clustering algorithm is performed in the obstacle region to generate a list of object hypotheses, and their characteristics are used for the coarse identification. In module three, for the object hypotheses in front of the vehicle, particular objects of interest, including vehicles, pedestrians, motorcycles and bicycles, are identified using a multi-class object detector with deformable part-based models, and tracked using particle filters. In the experiments, the data of various typical but challenging road scenarios were acquired by a Velodyne sensor and a monocular camera, and the results have demonstrated the effectiveness of the proposed system.