An Automatic Framework for Detecting and Characterizing Performance Degradation of Software Systems

Software systems that run continuously over a long time have been frequently reported encountering gradual degradation issues. That is, as time progresses, software tends to exhibit degraded performance, deflated service capacity, or deteriorated QoS. Currently, the state-of-the-art approach of Mann-Kendall Test & Seasonal Kendall Test & Sen´s Slope Estimator & Seasonal Sen´s Slope Estimator (MKSK) detects and characterizes degradation via a combination of techniques in statistical trend analysis. Nevertheless, we pinpoint some drawbacks of MKSK in this paper: 1) MKSK cannot be automated for large scale software degradation analysis, 2) MKSK estimates the degradation trend of software in an oversimplified linear way, 3) MKSK is sensitive to noise, and 4) MKSK suffers from high computational complexity. To overcome all these limitations, we propose a more advanced approach called Modified Cox-Stuart Test & Iterative Hodrick-Prescott Filter (CSHP). The superiority of our CSHP approach over MKSK is validated through extensive Monte Carlo simulations, as well as a real performance dataset measured from 99 real-world web servers.