Handling Large Datasets in Parallel Metaheuristics: A Spares Management and Optimization Case Study

Parallel metaheuristics based on Multiple Independent Runs (MIR) and cooperative search algorithms are widely used to solve difficult optimization problems in diverse domains. A key step in assessing and improving the speed of global convergence of parallel metaheuristics is tracing solutions explored by the MIR-based algorithm. However, this generates large amounts of data, thus posing execution problems. This problem can be resolved by using a flow control workflow to govern the execution of the MIR-based parallel metaheuristics. Using a Spares Management and Optimization case study for the logistics industry, this paper analyzes the performance of the flow control workflow with different problem sizes. We show that by appropriately setting workflow parameters, namely: (1) stop criterion to limit the amount of data cached and exchanged, and (2) clustering policy to distribute/aggregate parallel processes to compute nodes selectively, the performance of the algorithm can be improved.