Extensive analysis of the Kad-based distributed computing system DuDE

The distributed computing of data is a challenging task in terms of the self-organizing task distribution and computing, especially if distributed computing systems are becoming very large and complex. Therefore, the distributed hash table (DHT)-based P2P system called DuDE has been developed to compute statistics of access nodes of Internet service providers in an efficient way. DuDE exploits the high failure resilience and scalability features of the DHT network Kad to achieve a high-performance distributed system, which avoids the bottlenecks of a centralized computing solution. To ensure highly available data, Reed-Solomon codes for reliable distributed data storage are utilized. For implementing DuDE, usual working steps of distributed computing have been extended to realize a highly scalable computing system. We have developed a simulation model for a large-scale DuDE network consisting of up to 9,000 access nodes for computing statistics. In this paper, simulation results are presented, which demonstrate that DuDE is able to almost linearly accelerate the distributed computing compared to a centralized solution while introducing low traffic overhead.