Experiences with an AR evaluation test bed: Presence, performance, and physiological measurement

This paper discusses an experiment carried out in an AR test bed called “the pit”. Inspired by the well-known VR acrophobia study of Meehan et al. [18], the experimental goals were to explore whether VR presence instruments were useful in AR (and to modify them where appropriate), to compare additional measures to these well-researched techniques, and to determine if findings from VR evaluations can be transferred to AR. An experimental protocol appropriate for AR was developed. The initial experimental findings concern varying immersion factors (frame rate) and their effect on feelings of presence, user performance and behavior. Unlike the VR study, which found differing frame rates to affect presence measures, there were few differences in the five frame rate modes in our study as measured by the qualitative and quantitative instruments, which included physiological responses, a custom presence questionnaire, task performance, and user behavior. The AR presence questionnaire indicated users experienced a high feeling of presence in all frame rate modes. Behavior, performance, and interview results indicated the participants felt anxiety in the pit environment. However, the physiological data did not reflect this anxiety due to factors of user experience and experiment design. Efforts to develop a useful AR test bed and to identify results from a large data set has produced a body of knowledge related to AR evaluation that can inform others seeking to create AR experiments.