Predicting and Evaluating Distributed Communication Performance

Application of hardware-parameterized models to distributed systems can result in omission of key bottlenecks such as the full cost of inter- and intra-node communication in a cluster of SMPs. However, inclusion of message and middleware characteristics may result in impractical models. Nonetheless, the growing gap between memory and CPU performance combined with the trend toward large scale clustered shared memory platforms implies an increased need to consider the impact of middleware on distributed communication. We present a software-parameterized model of point-to-point communication for use in performance prediction and evaluation. We illustrate the utility of the model in two ways: 1) to derive a simple, useful, more accurate model of point-to-point communication in clusters of SMPs, 2) to predict and analyze point-to-point and broadcast communication costs in clusters of SMPs. We present our results on an IA-64-based cluster.