A Synchronization-Induced Checkpoint Protocol for Group-Synchronous Parallel Programs

Group check pointing is a fix between global check pointing and log-based recovery. It features both reduced runtime overhead and localized recovery effect for improving the fault-tolerance performance of large-scale distributed systems. However, parallel programs cannot efficiently benefit from this strategy, as they often involve synchronous or semi-synchronous interactions that incur extra idling delays between processes as well as between process groups. This paper presents an analytical study on such delays and the corresponding delay optimization strategies. Observing that certain parallel programs exhibit patterns of "synchronization groups", we develop a Synchronization-Induced Checkpoint protocol that manages checkpoints around such groups. The protocol keeps advantages of ordinary group check pointing, and meanwhile minimizes the costs of synchronization-induced delays. We also broadly categorize the known synchronization patterns and establish their relations with suitable checkpoint strategies for parallel programs.