Scalable duplicate pruning strategies for parallel A* graph search

In parallel A* graph search on distributed-memory machines, different processors may perform significant duplicated work if inter-processor duplicates are not pruned. The only known method for duplicate pruning associates a particular processor with each distinct node of the search space using a suitable hash function. Then duplicate nodes arising in different processors are transmitted to the same processor, and thereby pruned. There are two main drawbacks attributable to such an approach: (1) Load balance is determined solely by the hash function and is unsatisfactory. (2) Node transmissions for duplicate pruning are global; this can lead to hot spots in the network. We propose two different duplicate pruning techniques that outperform this hashing-only method by using: (1) separate algorithms for duplicate pruning and load balancing, and (2) a novel search space partitioning scheme that evenly spreads out the bandwidth requirement for pruning over the entire parallel architecture. Using the Traveling Salesman Problem (TSP) as a test case, we find that on a 10-dimensional nCUBE2 hypercube multicomputer, our pruning strategies yield a speedup improvement of more than 135% over previous methods that do not prune any duplicates, and more than 155% over the hashing-only pruning scheme