Improving Automated Testing of Multi-threaded Software

This paper discusses an approach to avoid incorrect results in the execution of automatic tests of multi-threaded systems. We argue that such incorrect results have two main sources. First, it is typically difficult to determine when all threads have finished processing and thus when it is safe to perform the test assertions. Second, background threads can change the system state while assertions are being performed, thus producing non-deterministic results. The main contributions of this work are: (i) a generic approach that ensures that test assertions are performed in a safe moment; (ii) implementation details of such an approach using aspect-oriented programming (AOP); and (Hi) an evaluation of the proposed approach.