Load balancing and dynamic scaling of cache storage against zipfian workloads

We investigate near-optimal load balancing and auto-scaling solutions that can get the best delay performance while attaining high utilization in two-tier cloud storage systems. The first tier consists of a distributed cache composed of leased resources from a computing cloud (e.g., Amazon EC2) and the second tier consists of a persistent distributed storage (e.g., Amazon S3). At any point in time our solutions dynamically determine how many cache servers should be active as the incoming load, object popularities, and service rate of persistent storage are subject to change. We first present a load balancing solution for a given set of cache servers that can evenly distribute the load against workloads with an unknown Zipf distribution while keeping the overall cache hit ratios close to the maximum. Building on this load balancer, we then present our auto-scaling solution that uses the request backlogs in the caching tier to determine the number of active cache servers to be used in the next epoch. Via simulations, we show that the end-to-end design is quite effective in converging to the right amount of service capacity against the system dynamics.