Exploiting delayed synchronization arrivals in light-weight data parallelism

SPMD (single program multiple data) models and other traditional models of data parallelism provide parallelism at the processor-level. Barrier synchronization is defined at the level of processors where, when a processor arrives at the barrier point early and waits for others to arrive, no other useful work is done on that processor. Program restructuring is one way of minimizing such latencies. However, such programs tend to be error-prone and less portable. The author discusses how multithreading can be used in data parallelism to mask delays due to application irregularity or processor load imbalance. The discussion is in the context of Coir, the object-oriented runtime system for parallelism. The discussion concentrates on shared memory systems. The sample application is an LU factorization algorithm for skyline sparse matrices. The author discusses performance results on the IBM PowerPC-based symmetric multiprocessor system