Vision-based robust calibration for optical see-through head-mounted displays

We propose Vision-based Robust Calibration (ViRC) method for OSTHMDs equipped with a camera. In the ViRC method, calibration parameters are decomposed into off-line parameters that remain constant relative to the positional relationship between the camera and the virtual screen, and on-line parameters related to the user´s eye. Calculating the off-line parameters beforehand reduces the number of unknown parameters in the on-line phase, giving robust protection against the user´s misalignments during calibration. In the off-line phase, the approximate position of the user´s eye is calculated using the PnP algorithm. In the online phase, the actual position of the user´s eye is estimated from the approximate one by non-linear minimization. In our experiments, we show that the ViRC method can decrease reprojection error by as much as 83% compared with the conventional method based on the DLT algorithm.