Advanced obstacles detection and tracking by fusing millimeter wave radar and image sensor data

Reliable object detection and recognition for Adaptive Cruise Control (ACC) by multi-sensor and multi-algorithm have attracted much attention recently. Aiming at the calibration complexity problem, a novel calibration method is proposed to calibrate the millimeter wave radar data and the CCD camera data using a homography. The proposed method does not require estimation of rotation and translation between them or intrinsic parameters of the camera. And the calibration can be done without any manual manipulation. Tracking of the detected object by radar is done in image plane as the radar is insensitive to object size and has a low object recognition rate as well. A new tracking algorithm based on Mean Shift(MS) algorithm with Unscented Kalman filter (UKF)is proposed to track the detected objects in the image plane, in which UKF is used to predict the initial position of iteration for MS algorithm. Then MS is applied to find out the accurate position of the object and positions of objects are mapped to radar plane to judge it is a new object or old one. We demonstrated the validity of the proposed methods through experiments.