Multi-rate fusion of visual and inertial data

We present a method for fusing data from a vision system and an inertial measurement unit. The problem considered is that of pose estimation for a rapidly moving camera, and we show that given delayed low bandwidth visual observations of line correspondences and high bandwidth rate gyro measurements we are able to estimate the camera orientation with respect to an inertial frame. This can be done without knowing or estimating the camera position. The estimates are of high bandwidth and this is consistent with real-time constraints due to the complementary characteristics of the two sensors which are fused in a multi-rate way. The algorithm has interesting mathematical properties as it does not use any local parameterization of the orientation such as Euler angles. Instead, the state estimates evolve on the group of rotation matrices.