Towards a Comprehensive Test Suite for Detectors of Design Patterns

Detection of design patterns is an important part of reverse engineering. Availability of patterns provides for a better understanding of code and also makes analysis more efficient in terms of time and cost. In recent years, we have observed a continual improvement in the field of automatic detection of design patterns in source code. Existing approaches can detect a fairly broad range of design patterns, targeting both structural and behavioral aspects of patterns. However, it is not straightforward to assess and compare these approaches. There is no common ground on which to evaluate the accuracy of the detection approaches, given the existence of variants and specific code constructs used to implement a design pattern. We propose a systematic approach to constructing a comprehensive test suite for detectors of design patterns. This approach is applied to construct a test suite covering the Singleton pattern. The test suite contains many implementation variants of these patterns, along with such code constructs as method forwarding, access modifiers, and long inheritance paths. Furthermore, we use this test suite to compare three detection tools and to identify their strengths and weaknesses.