Cache Performance and Efficiency Factors of Parallel Data Structures

Multi-core CPUs are very efficient at executing multiple threads at the same time without significant performance penalty; this capability, however, results in increasing demand for the memory and the caches, which not only have to serve multiple parallel requests but also have to endure the consequences of parallel programming. The performance of parallel applications is not only limited by the CPU and the internal level of parallelism that the algorithms and data structures allow, but is also restricted by their memory characteristics. Altering the data structure for example to work with machine word-sized pointers required by atomic operations incurs additional cache misses, while a lock protecting a critical section not only consumes memory, but could also be responsible for increased memory traffic for cache-line invalidations when acquired or released. We investigate these effects and analyze the behavior of different parallelization mechanisms, both blocking and lock-free solutions, through the example of a basic data structure: a hash table.