Social semiotic analysis of the design space of augmented reality

Text is one of the multimodal components used in many augmented reality (AR) applications. Fulfilling the design goals of an application depends, to a large extent, on the successful design of these components. This paper examines the role of text in AR user interfaces from a social semiotic and linguistic perspective. Since analysis of the text in AR applications has mostly focused on aspects such as legibility, placement, occlusion-related issues and exploration of methods to overcome other visual challenges, we suggest that such a social semiotic and linguistic analysis of the textual content can provide insights into user interface and application design for AR researchers and developers. Our analysis suggests that the role of text in AR can differ, and depends, at least in part, on the type of text-image interaction of which it is a part.