Graphical annotation as a visual language for specifying generalization relations

This paper explores how graphical annotation can be used as a visual language for specifying interpretations of user actions in an environment for programming by example. Attaching text labels to graphical elements is a natural visual notation that appears in many kinds of hand drawn diagrams, such as those appearing in user manuals, to indicate part-whole relationships. Programming by example systems have the problem that each user action, such as clicking on a graphical object, is potentially ambiguous. When the domain of the system is extended, the user must communicate to the system what the intended interpretation of newly introduced objects should be. The author presents a technique for allowing an end user to add graphical annotation to a drawing to indicate the meaning of its parts