On the principles and factors determining the predictive success of ecosystem models, with a focus on lake eutrophication models

If an ecosystem model is tested against an independent set of data, then the achieved r2-value (when modelled values are compared with empirical data (y)) will depend on the uncertainty in the empirical y-value (i.e. the uncertainty in the y-direction) and on the structuring of the model, i.e. which processes and model variables are accounted for and the empirical uncertainty of the model variables (i.e. the uncertainty in the x-direction). The aim of this work is to highlight fundamental principles and factors regulating the predictive success of ecosystem models. Three r2-values are discussed: (1) the theoretically highest reference r2 (rr2), which is defined in this paper as: rr2=1−0.66×CV2, where CV is the characteristic coefficient of variation for the given y-value (CV=S.D./MV; S.D.=standard deviation; MV=mean value); (2) the empirically-based highest r2 (re2), which is determined from a regression when two empirical data sets of y are compared (Emp1 vs. Emp2), and (3) the highest achieved r2 when modelled y-values are regressed against empirical y-values (r2). Ecosystem models can never be expected to yield a high r2-value if the target variable (y) cannot be empirically determined well and/or if the model is based on driving variables (x) yielding high coefficients of variation (CV). This work also gives a compilation of characteristic CV-values for many standard lake variables. Monte Carlo simulations (uncertainty analyses) have been carried out to illustrate the practical use of the results using examples with lake eutrophication models. For phosphorus concentrations in lakes, it has been shown that the highest empirically-based r2, re2 is 0.85. This value emanates from data from 25 Swedish lakes and is highly dependent on the number, quality and range of the data. The theoretically highest reference r2-value to predict phosphorus concentrations in lakes depends on the characteristic CV-value for phosphorus in lakes, which is 0.35, and hence rr2=0.92. The lake eutrophication model which has achieved the highest r2-value when tested against the widest range of empirical data is the OECD model (OECD, 1982), which gave an r2 of 0.86. This models is, however, very simplistic and cannot be used to address many important lake management issues. The only way to achieve the highest theoretical reference r2 of 0.92 for phosphorus concentrations in lakes using mechanistically-based, dynamic models is to structure the model in such a way that it includes no more no less than the most fundamental processes regulating the phosphorus fluxes (like internal loading, stratification and seasonal variation). Such models must also be based on driving variables with a minimum of inherent empirical uncertainties (i.e. small CV-values), and not the opposite, as for most lake eutrophication models used today based on the mean concentration of phosphorus in tributaries, which is one of the most variable of all model variables for lake eutrophication models.