A Self-Organized Load-Balancing Algorithm for Overlay-Based Decentralized Service Networks

A service network with decentralized ownership is a system where nodes offering a variety of services are administered by different organizations -- or even by a set of individuals. In such a context, nodes hosting services can dynamically enter and exit the system without prior notice, and there is no centralized point of control. If one wants to build into such a system the ability to direct incoming requests for the various hosted services to those nodes that can efficiently fulfill them, one option is to introduce in the system an entity that serves as a gateway to accept service requests, and is an intermediary to re-direct requests as needed. That implies that this intermediary is able to acquire and maintain accurate and up-to-date information on where it can direct incoming requests. Another option, which is the one we pursue in this paper, is to build the system as an overlay network, in which the nodes hosting instances of each of many different types of services can self-organize as "virtual clusters", and efficiently load-balance incoming requests amongst themselves. We describe our design and evaluation of a decentralized computing framework of this kind. We leverage a resilient peer-to-peer overlay that automatically re-configures its topology, responding to the number of different service types executing on the peer nodes, the dynamics of the participation of those nodes (peer churn), and the traffic coming into the system for the various services.