Improving the performance of OLTP workloads on SMP computer systems by limiting modified cache lines

Symmetric multiprocessor (SMP) computer systems with more than four CPUs often exhibit significantly lower overall performance than would be expected from the sum of the performance of the individual CPUs. One of the causes of this degradation is the increased average memory latency due to cache to cache migration of modified cache lines. Such transfers often incur significantly longer latencies than a simple cache miss, which can be satisfied from main memory. By setting an upper bound on the number of modified cache lines that are allowed to exist in a main memory when this limit is exceeded, the average memory latency and overall system performance on an online transaction processing (OLTP) workload can be improved. This paper presents an investigation of this concept, called original limiting, on a commercial SMP system. The Original Limiting concept was implemented in the second level cache (SLC) of the Unisys NX6830 series of SMP systems, which support up to eight CPUs. An original limiting queue (OLQ) was added to limit the number of exclusive or modified lines in a 5% improvement in the number of transactions processed per minute, by reducing the average memory latency. A variety of experiments indicate that the OLQ is a simple, but effective, mechanism to enhance the performance of OLTP applications on SMP systems.