Continuous natural user interface: Reducing the gap between real and digital world

Augmented reality (AR) presentation enables the creation of natural user interfaces that employ the whole user´s environment as interaction device. Additionally, by using hand based 3D interaction with gestures that have a physical meaning like grabbing, dragging, and dropping this leads to a user experience that is intuitive, since close to the real world´s behavior. We propose a novel approach to an AR-based natural user interface, that goes one step further by enabling the contents of the interface to switch domains from a virtual instance in AR to a physical instance in the real-world. All instances stay associated and changes made to the physical instance will be reflected on the virtual one. Because the behavior of our interface in AR is in key aspects consistent with the real-world, the gap between those domains is made less salient. To demonstrate our concept, we have implemented an exemplary industrial use case. Our main contribution is the methodology for an intuitive interface we call continuous natural user interface (CNUI). Additionaly, we conducted a user study to investigate the acceptance of this kind of interface. Results indicate an ergonomic ease and after a training period also an increased performance when using our system.