Camera and lidar fusion for pedestrian detection

The aim of this work is to propose a fusion procedure based on lidar and camera to solve the pedestrian detection problem in autonomous driving. Current pedestrian detection algorithms have focused on improving the discriminability of 2D features that capture the pedestrian appearance, and on using various classifier architectures. However, less focus on exploiting the 3D structure of object has limited the pedestrian detection performance and practicality. To tackle these issues, a lidar subsystem is applied here in order to extract object structure features and train a SVM classifier, reducing the number of candidate windows that are tested by a state-of-the-art pedestrian appearance classifier. Additionally, we propose a probabilistic framework to fuse pedestrian detection given by both subsystems. With the proposed framework, we have achieved state-of-the-art performance at 20 fps on our own pedestrian dataset gathered in a challenging urban scenario.