Issues in Communication Heterogeneity for Message-Passing Concurrent Computing

Heterogeneity in interconnection network throughput and latency has recently become a major issue for parallel computing applications. With the universal prevalence of multicore processors, large scale clusters and, most critically, cloud platforms, variations in communication characteristics of orders of magnitude are possible within a single execution environment. When applications also exhibit heterogeneity and irregularity in their communication patterns, process placement can make the difference between good and unacceptable performance. We discuss techniques for analyzing and addressing these factors in the context of a computational fluid dynamics application for the study of blood flow, on typical parallel platforms: a local parallel machine, a workstation network, and IaaS cloud-based cluster. Our experiences show problem sizes and platform sizes for which communication variations have significant impact, and suggest methods for process mapping that are likely to alleviate the detrimental effects of communication heterogeneity in different environments.