Measuring the Effectiveness of Self-Healing Autonomic Systems

Benchmarks are a central force in engineering progress, providing the objective ability to quantify improvement and justify design decisions. In previous work, we brought together the concepts of benchmarks and autonomic computing, describing a vision of benchmarks that quantitatively evaluate a computing system along the four core autonomic dimensions of self-healing, self-configuration, self-optimization, and self-protection. In this paper, we describe our experience with implementing a practical benchmark for the self-healing dimension of autonomic capability, which goes beyond simple measures of fault tolerance by including a measure of autonomic maturity. Our benchmark is capable of quantifying the autonomic self-healing capability of complex, production-scale enterprise solutions based on J2EE middleware (and indeed is currently being used for such purposes)