Towards Autonomous Air-to-Air Refuelling for UAVs using visual information

For the purposes of achieving Autonomous Air-to-Air Refuelling (AAAR) in Unmanned Aerial Vehicles (UAVs), this paper presents the use of a visual tracking algorithm based on direct methods and image registration techniques, with the aim of solving the drogue tracking problem in order to obtain vision-based relative position estimations of the aircrafts for the probe and drogue refuelling method. Proposed vision-based AAAR approaches to date have explored the use of features (such as corners, painted marks, or LEDs) to detect and estimate the relative motion of either the receiver or the tanker aircraft, with the drawback that sometimes this requires the installation of specific hardware on-board. Conversely, the strategy we propose to use does not require the installation of additional hardware on-board. The strategy is based on a hierarchical implementation of an image registration technique: the Inverse Compositional Image Alignment ICIA. Real images and real flight hardware (probe and drogue) are used to test the algorithm using a robotic testbed that simulates the motion of the aircrafts (the tanker and the receiver) during the refuelling task. Results show that the tracking algorithm is robust to fast motion, changes in appearance, and situations where part of the drogue is occluded or is outside of the field of view of the camera. Additionally, results show that robust position estimations at real-time frame rates are obtained, proving that the technique is fast enough to form the basis for automated aerial refuelling sensing capabilities.