Request redirection algorithms for distributed Web systems

Replication of information among multiple servers is necessary to support high request rates to popular Web sites. We consider systems that maintain one interface to users, even it they consist of multiple nodes with visible IP addresses that are distributed among different networks. In these systems, first-level dispatching is achieved through the Domain Name System (DNS) during the address lookup phase. Distributed Web systems can use a request redirection mechanism as second-level dispatching because the DNS routing scheme has limited control on offered load. Redirection is always executed by the servers, but there are many alternatives that are worth investigating. We explore the combination of DNS dispatching with redirection schemes that use centralized or distributed control on the basis of global or local state information. In fully distributed schemes, DNS dispatching is carried out by simple algorithms because load sharing is taken by some redirection mechanisms that each server activates autonomously. On the other hand, in fully centralized schemes, redirection is used as a tool to enforce decisions taken by the same centralized entity that provides the first-level dispatching. We also investigate hybrid strategies. We conclude that distributed algorithms are preferable over their centralized counterpart because they provide stable performance, take content-aware dispatching decisions, limit the percentage of redirected requests, and their implementation is much simpler than that required by centralized schemes.