Measurement and Interpretation of Micro-benchmark and Application Energy Use on the Cray XC30

Understanding patterns of application energy use is key to reaching future HPC efficiency goals. We have measured the sensitivity of energy use to CPU frequency for several microbenchmarks and applications on a Cray XC30. We suggest first order models for performance and power vs. frequency and show that these are sufficient to accurately fit the measured energy data. Examination of the resulting energy model shows that an application´s energy/frequency profiles have minima only if a) the frequency change crosses an architectural balance point that is performance-critical for the particular application or b) a significant fraction of the runtime is spent in off-chip operations or c) there is sufficient static power drawn to motivate a race-to-halt. All three forms of energy minima are represented in our sample of HPC applications. The energy-optimal frequencies on this architecture are: MILC (1.8 GHz), GTC (3.6 GHz) and Mini-DFT (1.6-1.8 GHz).