A multiphase partitioner and scheduler for distributed memory systems

Most of the current loop partitioning methods for distributed memory systems first partition program data (either automatically or through user specified distributions) and then use `owner computes´ rule to to localize the underlying references. Since parallelism is an inherent characteristic of the code, for aggressive optimizations, one should take an orthogonal approach of first partitioning the code to maximize the parallelism and then allocate data appropriately. We describe a multiphase partitioner and scheduler based on the above motivation which consists of five phases. First, the code partitioning phase determines a set of directions for minimizing the communication by trading the parallelism to a certain extent. Next, the data distribution phase attempts to achieve computation+communication load balance by distributing the underlying data. The granularity adjustment phase attempts to further eliminate communication to minimize completion time. Finally, the load balancing phase attempts to reduce the number of processors without degrading the completion time and the mapping phase schedules the partitions on available processors