Optimal communication algorithms for heterogeneous computing over ATM networks

We present efficient algorithms for designing optimal collective communication primitives for cluster-based heterogeneous computing across ATM networks. Specifically, we consider the following four collective communication problems that occur frequently in parallel computing: one- to- all broadcast, all-to-all broadcast, one-to-all scatter and one-to-all scatter. We formulate these problems as combinatorial optimization problems for finding a system of virtual path routes in the network, with the objectives of minimizing the path lengths, the link congestions and the node copy loads. We consider two cases of data sources, namely homogeneous and heterogeneous. For the case of homogeneous sources, we present polynomial-time algorithms for these problems by converting them to the maximum-flow problems. For the case of heterogeneous sources, we show that these problems are NP-complete. Finally, we present novel randomized algorithms for there problems, and show that with high probability there algorithms yield good solutions in the sense that the objective functions yield values close to the optimums. These randomized algorithms make use of linear programming and are easy to implement, which makes them a potential candidate for practical implementations, even for very large networks