Trace visualization for program comprehension: A controlled experiment

Understanding software through dynamic analysis has been a popular activity in the past decades. One of the most common approaches in this respect is execution trace analysis: among our own efforts in this context is EXTRAVIS, a tool for the visualization of large traces. Similar to other trace visualization techniques, our tool has been validated through anecdotal evidence, but should also be quantitatively evaluated to assess its usefulness for program comprehension. This paper reports on a first controlled experiment concerning trace visualization for program comprehension. We designed eight typical tasks aimed at gaining an understanding of a representative subject system, and measured how a control group (using the Eclipse IDE) and an experimental group (using both Eclipse and EXTRAVIS) performed in terms of correctness and time spent. The results are statistically significant in both regards, showing a 21% decrease in time and a 43% increase in correctness for the latter group.