The impact of synchronization and granularity on parallel systems

A study is made of the impact of synchronization and granularity on the performance of parallel systems using an execution-driven simulation technique. It is found that, even though there can be a lot of parallelism at the fine-grain level, synchronization and scheduling strategies determine the ultimate performance of the system. Loop-iteration-level parallelism seems to be a more appropriate level when those factors are considered. Barrier synchronization and data synchronization at the loop-iteration level are also studied. It is found that both schemes are needed for a better performance