Perspective distortion rectification for planar object based on LIDAR and camera data fusion

Planar objects like road surface, traffic sign, and street sign, are ubiquitous in traffic scenes. Such objects provide important information for driving safety. But it is challenging to reliably detect and recognize them by using camera in complex environments. One of the main challenges comes from the perspective distortion. Substantial algorithms have been proposed to address this problem for specific planar objects by taking advantage of the characteristics of the problems at hand. In contrast to those target-specific algorithms, we propose a generic algorithm to rectify the perspective distortion of planar objects based on the data fusion of LIDAR and camera. The main idea of our algorithm is to use LIDAR data to recover the 3D geometric model of a planar object, and then pose a virtual camera to look orthogonally toward it. This virtual camera is capable of generating the desired fronto-parallel view of the planar target. Meanwhile, the rectifying homography between the virtual and real cameras only relies on the plane parameters and the relative pose between both cameras, irrespective of the category of the planar object. This enables our algorithm can be applied to various planar objects with arbitrary shapes. Our experimental results show that the proposed algorithm is effective and efficient to correct the perspective distortion of common planar objects in traffic scenarios.