SCALLOP: a scalable and load-balanced peer-to-peer lookup protocol for high-performance distributed systems

Many large-scale servers are implemented in a peer-to-peer distributed system. Achieving rapid response time in such a system relies on a scalable lookup protocol to efficiently locate the requested items and a load-balancing mechanism to avoid the hot spot problem by evenly distributing lookup requests. We present a peer-to-peer lookup protocol that addresses both issues. Our lookup protocol uses the technique of distributed hash table (DHT) to store in each node O(logN) routing information in an N-node distributed system. Unlike other DHT-based peer-to-peer protocols, our protocol constructs a balanced lookup tree to avoid hot spots. We compare our protocol with a commonly used peer-to-peer lookup protocol. The experimental results show that our protocol reduces up to 51% of the lookup requests on heavily loaded nodes. Furthermore, our protocol reduces the total lookup requests and thus delivers better performance.