On improving latency of geographically distributed key-value stores via load balancing with side information

We propose novel load balancers for key-value data stores running as a service over geographically distributed data centers. We assume that the key-value store has the following properties: (i) The requests (e.g., put/write, get/read, copy, etc.) can be directed to any one of the cluster nodes regardless of which cluster nodes store the information; (ii) Different objects can have different number of replicas and applications can set their desired consistency levels; and (iii) it is possible for intermediate (authorized) proxies to query/learn cluster topology and key space to node mappings. We propose two load balancing strategies that would be useful under these assumptions: (i) Static strategy that makes assignment decisions solely based on where the majority of the hosting nodes are located for a given key. (ii) Dynamic strategy where proxy nodes actively probe the performance of different cluster nodes as a function of their location (i.e., which data center they belong) and key-specific operational parameters (e.g., consistency level and replication factor) to select the best cluster node to serve each request. Our evaluations show that 2× gains for average read/write delays against random load balancers, while dynamic approach significantly outperforming the static approach between 60th and 90th percentiles.