The twin bottleneck effects [parallel algorithms]

For a class of parallel algorithms common in scientific applications, the authors have derived equations that determine idle and elapsed execution time, and account for overlap of communication and task execution. An unusual effect has also been observed. The speed of these algorithms is not solely dependent on the most computationally loaded processor, but instead depends on the two most loaded processors. The authors call this the twin bottleneck effect. They present the class of algorithms described by the model and find equations for idle and elapsed execution times on individual processors and the elapsed time for the total ensemble. The equations encompass an arbitrary number of processors and iterations, and each processor can be assigned a unique workload and communication time. The authors describe the twin bottleneck and the conditions under which it arises, and discuss ways the equations can be applied to real world situations to minimize optimization efforts