Self-Optimization of a DHT-Based Discovery Science

Discovery in large, highly dynamic service oriented systems cannot be realized by traditional means, i.e. with a centralized index. Instead, distributed data structures are used, with a distributed hash table deployed on an overlay network being a typical example. The efficiency of overlay networks built on top of the IP network commonly suffers from the mismatch between the topologies of the overlay and the underlying IP network, resulting in unnecessary traffic and increased latencies. Substantial improvement can be achieved by optimizing the logical links between overlay nodes. In this paper, we propose a new method for self-optimization of a pervasive, DHT-based discovery service. Our method has no need for active measurement of inter-node latencies, thus minimizing network traffic costs of node insertion and topology maintenance. We verify our method by means of analysis of a large data sets of latency measurements between arbitrary nodes on the Internet, proving correlation among common IP prefix length of communicating nodes and latency.