Golden-run alignment for analysis of robustness testing results: dealing with diagnostics issues

For the analysis of robustness testing results, comparison with a golden run is commonly used to determine whether robustness failures occurred or not. One limitation of this approach is that traditional comparison techniques require the system to have a repeatable behavior under the same workload, whether or not in presence of faults. This limits the applicability of this technique, since most systems have indeterminist behavior due to concurrency, for example. Moreover, tools that use golden-run comparison hardly ever give information about what are the common patterns of behavior between the golden-run and a faulty trace and where there are deviations due to action of fault-tolerance mechanisms, for example. In this paper, we show how the use of sequence alignment algorithms, from computational biology, can be of help. Besides allowing the determination of regions of similarities between two sequences that are not exactly equal, these algorithms present results in a visual form, highlighting the different zones where there are common patterns between the two sequences. We also point out other ways in which alignment algorithms can be useful as well.