Predicting Parameter Sweep Jobs: From Simulation to Grid Implementation

Efficiently using the computational power made available through desktop grids based distributed systems is a complicated and many-sided problem, caused by the intermittent resource availability. In this paper a novel solution is presented for predicting the runtimes of parameter sweep jobs. These jobs are characterized by their lack of inter-dependence and suitability for runtime prediction by modeling. This makes them ideal candidates for deployment on volatile grid configurations using prediction based techniques. The parameter sweep prediction framework used to make the predictions is referred to as GIPSy (grid information prediction system). Previous research involving GIPSy has focused on results obtained during simulation were it is necessary to make some basic assumptions. By combining GIPSy with PGS (prediction based grid scheduling), an actual grid implementation, real results can be obtained. A detailed comparison between the expected results, based on simulation analysis, and the final results is given. Discrepancies are highlighted and possible causes are identified, solutions are proposed and implemented. By comparing the results for different model building configurations an optimal configuration is found that produces reliable result independent of the chosen job type. Results are presented for a quantum physics problem and two simulated workloads represented by sleep jobs.