On assessing self-adaptive systems

Self-managed communication systems are using self-adaptive algorithms to control the services they offer. It is assumed that autonomy of system operation helps to reduce operational expenses, to increase performance or even to allow the emergence of new functionalities. The comparison and the understanding of operation (jointly termed here ¿assessment¿) of different self-managed systems require their complete description. In addition to traditional performance metrics (which these new systems still need to satisfy) this comparison must include new characteristics based on their adaptive nature. Due to the huge amount and the variety of operational conditions assumed for these systems the focus should be on the abilities of systems to solve problems rather than on their exhaustive testing. We describe this ability to solve a problem as a new metric, We will also present the process to determine the metric based on a series of performance measurements. We demonstrate that the metric helps appraise the ability of a system to cope with new and never experienced situations and to build confidence in autonomous system operation.