Scaling OLTP applications on commodity multi-core platforms

Multi-core processor architectures can have significant performance advantage over traditional single core designs, which are limited by power and processor complexity. Predictions based on Moore´s Law state that a processor chip may accommodate thousands of cores in 5-10 years. Can software scale with the number of cores and achieve the performance potential? This paper uses two OLTP (online transaction processing) applications (TPCC-UVa and Sysbench-OLTP) as a case study to investigate this question and determine what the performance bottlenecks are. On an Intel 8-core platform, these applications (with slight modifications to run well on a many-core platform) achieve a speedup (in terms of the transaction throughput) of 3.68 and 5.26, respectively. To find the scalability bottlenecks the paper proposes a method based on function´s scalability value metric. Functions with a high scalability value limit the scalability. By looking at the functions with the highest scalability value across all functions in the kernel, libraries, and application processes, the paper finds that database buffer pool contention, database synchronization primitives, scheduler overhead and lock contention in System V IPC are the main bottlenecks for TPCC-UVa. In Sysbench-OLTP, database synchronization primitives and the kernel scheduler limit scalability. The paper also explores several ideas such as scalable database lock, scalable spin lock and RCU-based IDR API to improve the scalability.