Dealing with Test Automation Debt at Microsoft

At Microsoft, substantial time and resources are expended in test case development, execution and verification. Thousands of new tests are added to existing test suites without any kind of review regarding their unique contribution to test suite effectiveness or impact on test suite efficiency. This talk describes how we leverage existing code coverage data, together with reduction and prioritization techniques, to help each test team analyze its test suite and guide them in improving their suite´s effectiveness and efficiency. The analysis focuses on identifying and deprecating/prioritizing groups of tests cases, given specific tactical goals for example, increasing current test suite stability and reliability, better structuring of test suite migration efforts, reducing test suite execution time and testing with limited hardware resources.