Reprojection error as a new metric to detect assembly/disassembly maintenance tasks

Providing relevant information at the right time in the right place is the major challenge in augmented reality, especially when it is applied in industry related applications. Indeed, by superimposing virtual elements on images which capture the real scene, augmented reality has proved its potential and maturity for facilitating maintenance activities, especially in training, repairing or inspections. In this system, instructions for an assembly or disassembly actions are linearly displayed to users and triggered sequentially and manually by the operator once he has completed each individual step. In this paper, we explore a metric which will allow the system to determine automatically when two objects are assembled since it provides hints on the current step of the maintenance scenario. This metric is based on pose estimations and reprojection errors by considering that the two objects are independent. The first results obtained on both synthetic and real image sequences show that this metric is efficient in detecting assembly/disassembly instants. We also lend guidelines on how to integrate this metric in a bigger computer vision system designed around maintenance task scenarios provided using augmented reality.