Improving Search Speed on Pointer-Based Large Data Structures Using a Hierarchical Clustering Copying Algorithm

The increasing processor-memory performance gap makes improving the cache locality as important as the virtual memory locality. In many applications, especially in search algorithms on pointer-based large data structures, breadth-first copying algorithms increase cache misses, page faults and TLB misses. Since the depth-first copying only achieves limited locality improvement, several clustering copying algorithms have been proposed. In this paper, we propose "hierarchical clustering" which groups data objects at multiple hierarchical levels and provides better locality at both the cache and virtual memory levels of the memory hierarchy. We also propose a new copying algorithm for the hierarchical clustering, which uses multiple scan pointers and bounded workspace. Our copying algorithm almost always outperforms other algorithms; after copying, two representative microbenchmarks, employing a search tree and an array of associative lists, run approximately two (sometimes five) times faster than breadth-first copying.