Modeling visual interactive systems through dynamic visual languages

Visual interaction processes are modeled in this paper as sequences of visual sentences in which for each visual sentence only a limited set of user actions is possible. We introduce the notion of "dynamic visual language" as a weakly ordered set of visual sentences characterized by the presence of common elements. We present a formal model of derivation of visual sentences in a dynamic visual language in which each visual sentence specifies the possible actions which can be performed on it and the possible transformations it can go through. In this way, we offer a formal setting in which the interaction process can be formally specified. A user interface can be derived from the formal specification, so that it embeds proper context elements which limit user disorientation. The concepts are illustrated by the user interaction with a prototype of a digital library developed at the University of Bari.