Avoiding Tree Saturation in the Face of Many Hotspots with Few Buffers

In a multistage network, hotspots induce tree saturation. The known solutions employ a variety of techniques, including combining (which works only for certain kinds of messages), feedback damping (which appears to provide low utilization in the absence of hot spots), and large numbers of buffers. In practice, the approach used today is to provide large numbers of buffers: in a P-processor system, the rule of thumb appears to be to provide 10P buffers, but 10P buffers maybe too expensive for systems containing 10⁵ or more processors. Even employing Omega(P) buffers does not appear to provide any guarantees, however. We show that by organizing the switches so that the messages addressed to a particular processor can use only certain of the buffers, many hotspots can be tolerated with few buffers. For example, a switch with O(log P) buffers can tolerate a single hotspot with probability 1, and allows the first few hotspots to have a large number of buffers before being declared a hotspot. A switch with B buffers can be organized so that it blocks a particular non-hotspot message with probability less than O(1/s) if there are O(B/log s) hotspots, and can handle a factor of O(B (log log s)/log s) more hotspots before the probability becomes a constant. A similar approach can also be used to improve caching behavior in a multithreaded system in which one of the threads tries to consume all of the cache.