Filter-based calibration for an IMU and multi-camera system

Vision-aided Inertial Navigation Systems (vINS) are capable of providing accurate six degree of freedom (6DoF) state estimation for autonomous vehicles (AVs) in the absence of Global Positioning System (GPS) and other global references. Features observed by a camera can be combined with measurements from an inertial measurement unit (IMU) in a filter to estimate the desired vehicle states. To do so, the rigid body transformation between cameras and the IMU must be known with high precision. Extended Kalman filters (EKF) and Unscented Kalman filters (UKF) have been used to calibrate camera and IMU systems requiring only a simple calibration target and moderate IMU-camera motion. This paper focuses on indoor applications where it is assumed a user is able to easily manipulate the sensor package. We extend the UKF filter to calibrate an IMU paired with an arbitrary number of cameras, with or without overlapping fields of view.