Statistics-based ID management for load balancing in structured P2P networks

Structured peer-to-peer (P2P) networks are increasingly important in the design of distributed systems. However, due to their strict data placement rules, they are often prone to three main load imbalances: range, data, and execution skew. Today, many load balancing algorithms focus on range skew and assume the network data rate to be the bottleneck. In applications that focus on distributed request processing, like name resolution, those assumptions are not valid, since messages are typically small, but can produce significant load at the application level. Here, data and execution skew are most important and the bottleneck is the number of application requests a peer can process. As a solution for such scenarios, we propose a new load balancing algorithm based on ID management. It collects statistics of overlay link usage during normal operation and uses this information to provide suitable IDs to joining peers. Very little overhead is generated while the rate of successfully answered requests is improved by a factor of up to 3 in typical scenarios as shown by simulation. This work presents the first load balancing algorithm that can handle all three types of skew in scenarios that focus on processed application requests as the bottleneck.