Measuring and Understanding Policy System Service Rate Performnance

This paper examines the factors that significantly affect the latency performance of event-condition-action policy servers. In large scale applications, such as telecommunications QoS management, the offered traffic to a policy server can be expected to be hundreds, if not thousands, of events per second. Clearly, policy systems must exhibit throughput characteristics that are superior to the traffic demands placed on them. What makes the issue significant is the received learning from the AI literature concerning forward-chained rule-based systems. These systems, of which policy systems can be seen as a type, are reported to be notoriously slow and ineffective for real-time systems management. We investigate the similarities between policy systems and traditional AI rule-based systems, and examine in detail the reported cause for poor performance -the inference engine. Through experimentation and benchmarking we test the claims of the AI community in a policy system context. Encouragingly, we find that adequate policy system performance for large scale applications can be achieved and that the claims of poor system performance from the AI community concerning rule based systems do not in general apply. However, we do note that without thoughtful consideration, policy server throughput can be very poor and harmful to massive scale applications. Therefore, we note the factors which contribute to poor policy system performance.