Dense matrix factorization on MIMD multiprocessor architectures-some theoretical and empirical results

The authors discuss algorithms for parallel matrix factorization for dense matrices on distributed memory, MIMD (multiple instruction, multiple data) multiprocessor architectures based on asynchronous message passing, such as the hypercube and the ring. Two different task distribution patterns for matrix factorization have been analyzed and compared with respect to efficiency of processor utilization, speedup, and communication overhead. Empirical results on the Intel iPSC/2 Hypercube are presented. The hypercube architecture was found to have lower interprocessor communication overhead than the ring architecture for both block and wrap mapping. This was due to the lower diameter of the hypercube architecture for a given number of processors as compared to the ring architecture. For large matrices, however, the speedup and the efficiency of parallelism were found to be independent of the network diameter, whereas the interprocessor communication overhead was directly proportional to the network diameter