Multi-sensor data fusion for checking plausibility of V2V communications by vision-based multiple-object tracking

Automated vehicles require an adequate and reliable perception of the surrounding world in order to make good decisions. Using vehicle-to-vehicle (V2V) communication to exchange location data (i.e. time, position, heading and speed) can improve the perception beyond the capabilities of traditional on-board sensors (e.g. radar, lidar). However, it is vital to trust the data before it is being used. Cryptographic mechanisms can protect the exchange and authenticity of data but do not guarantee the correctness of the content. In this paper we present a vision-based multi-object tracking system for checking the plausibility of V2V communication. The system is addressing the challenge of fusing relative sensor observations as provided by a MobilEye vision-system with time-delayed absolute GNSS-based measurements from Cooperative Awareness Messages (CAMs) as provided by V2V. The plausibility check is implemented in a prototype and based on a state-of-the-art multiple-object tracking algorithm. The proposed system is evaluated and validated under real-world conditions by conducting several test drives under urban conditions.