Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances

In the context of a smart user assistance system for industrial manipulation tasks it is necessary to capture motions of the upper body and limbs of the worker in order to derive his or her interactions with the task space. While such capturing technology already exists, the novelty of the proposed work results from the strong requirements of the application context: The method should be flexible and use only on-body sensors, work accurately in industrial environments that suffer from severe magnetic disturbances, and enable consistent registration between the user body frame and the task space. Currently available systems cannot provide this. This paper suggests a novel egocentric solution for visual-inertial upper-body motion tracking based on recursive filtering and model-based sensor fusion. Visual detections of the wrists in the images of a chest-mounted camera are used as substitute for the commonly used magnetometer measurements. The on-body sensor network, the motion capturing system, and the required calibration procedure are described and successful operation is shown in a real industrial environment.