Rainbow: A Robust and Versatile Measurement Tool for Kademlia-Based DHT Networks

In recent years, peer-to-peer (P2P) file sharing applications have dominated the Internet traffic volumes, and among them, Bit Torrent and eMule constitute the majority. Bit Torrent and eMule deploy their distributed networks based on Kademlia, a robust distributed hash table (DHT) protocol, to facilitate the delivery of content. Kademlia-based DHT networks have intrigued researchers in P2P community to measure and analyze them. However, to the best of our knowledge, there is still not a well-designed crawler to carry out intensive measurement and analysis on them. In this paper, we develop Rainbow, a robust and versatile crawler for Kademlia-based DHT networks. For the first time, we theoretically analyze its convergence (a main issue of robustness), that is, Rainbow can complete the crawling within a limited time. Our analysis can also be applied to other P2P crawlers with the same sampling nature. Finally, we demonstrate that Rainbow can be applied as a versatile measurement tool to identify various characteristics of Kademlia-based DHT networks at a deep level.