Empirical Evaluation of Diagrams of the Run-time Structure for Coding Tasks

With object-oriented design, it is at least as important-possibly more important-to understand the runtime structure, in terms of objects and their relations, as to understand the code structure dealing with source files, classes and packages. Today, many tools and diagrams help developers understand the code structure. Diagrams of the runtime structure, however, are much less mature. One diagram of the run-time structure is a statically extracted, global, hierarchical Ownership Object Graph (OOG). The OOG conveys architectural abstraction by ownership hierarchy by showing architecturally significant objects near the top of the hierarchy and data structures further down. In an OOG, objects are also organized into named, conceptual groups called domains. We evaluate, in a controlled experiment, whether an OOG, as a diagram of the run-time structure, improves comprehension by giving developers the ability to distinguish the role that an object plays, not only by type, but also by named groups (domains) or by position in the run-time structure (ownership). We observed 10 participants, for 3 hours each, perform three feature implementation tasks on a framework application. Our results indicate that, on average, the OOG had a positive effect of varying extents on comprehension that reduced the time spent by 22%-60% and irrelevant code explored by 10%-60%. The difference was significant (p <; 0.05) for two of the tasks.