What do I need? A resource-based perspective on visual communication and interaction

Summary form only given. Any transformation process results from, and in, the use, production, consumption and transformation of some resources. Interaction processes are no exception to this, but the identification of the involved resources may present some challenges, as many of them are of an immaterial nature. In particular, when communication between a computer software and a human user occurs through the visual channel, the resources to be considered are not only the physical devices through which users manipulate and/or observe the state of the interaction, but also, for example, visual symbols at the interaction surface (as well as the available working space for drawing new symbols), data structures within the computing system (as in principle the whole state of the system might be relevant to the production of a system response), mental representations within the user´s mind (but possibly also levels of attention and tiredness depending on the overall state of the user), etc. Most importantly, the appropriate relations among all of these resources have to be maintained along a whole interaction. When Web-based multimedia or multimodal interaction is considered, new phenomena occur, as the temporal dimension of the interaction acquires relevance and configurations of resources may change due to events external to the interaction proper, as for example changes in bandwidth availability. These sources of complexity in the design of interactive systems are usually tackled by producing local models associated with the various viewpoints, usually expressed via different notations, as adopted in the communities to which stakeholders belong. If these viewpoints and models are not reconciled through proper systematic and formal methods, designs incur the risk of mismatches between the intended and the actual usages of the system, or effects of an interaction. The seminar will explore this arc of problems and some approaches to their solution, and will propose a u- ifying perspective for the resource-aware design of interactive systems and for the modelling of phenomena related to visual interaction.