Kullback-Leibler divergence as an estimate of reproducibility of numerical results

In large software projects using numerical solutions of equations, small changes in compiler options or parallelization methods can induce slight variations in the last digits of floating point numerical results, due for instance to the non-commutativity of operations. Unfortunate changes in the code (bugs) can induce even larger deviations in the results. We propose to use the Kullback-Leibler divergence estimated from the compression ratio of the results compared to reference results in order to automatically quantify those changes and automatize regression tests of numerical codes in software forges. We use the TDDFT PW-TELEMAN project as an example.