Distributed spanning tree algorithms for large scale traversals

The distributed spanning tree (DST) is an overlay structure designed to be scalable. It supports the growth from a small number of nodes to a large one. The DST is a tree without bottlenecks which automatically balances the load between its nodes. This sound paradoxale, but the DST breaks the common assumption that a tree is build of leaves and intermediate nodes. In a DST every nodes are equal. The nodes are put together into small cliques. Then, the cliques are put together into small cliques of higher level recursively. The cliques are represented in each node by a routing table. The memory space complexity of the routing tables is O(log(n)) for a n nodes DST. A theoretical description of the DST was already published but it does not provide enough information about the capacities of the DST. The purpose of this article is to give a practical view of what can be done with a DST. This document can be considered as a DST traversal catalog. It also outlines some characteristics and shares the lessons learned from our implementation errors.