QRP06-4: A Server Reassignment Algorithm for DHT Load Balance and the Effect of Heterogeneity

Application layer peer to peer (P2P) networks are considered to be the most important development for next generation Internet infra-structure. For these systems to be effective, load balancing among the peers is critical. Early structured P2P systems rely on the randomness of object ID generated with a consistent hash function to avoid the load imbalance issue, and has been known to result in an imbalance factor in the number of items stored at a node, even in the case of a homogeneous system where all nodes have the same capacity. We propose to address the problem systematically using an optimization based approach, and show that the problem of rearranging loads among the peers can be cast in the framework of a generalized assignment problem. The dual space local search (DSLS) heuristic algorithm is proposed, which minimizes the load movement involved in an efficient manner. We demonstrate the superior performance of our proposal in general, and its advantages over previous strategies. We also explore other important issues vital to the performance for the virtual server framework, such as the performance ramification of peer registration strategies. Significantly, we characterize systematically the effect of heterogeneity on load balancing algorithm performance, and the conditions in which heterogeneity may be easy or hard to deal with.