Fairplay in the verification of operational quantitative precipitation forecasts

The accuracy of weather forecasts is not only influenced by the skill of the forecasting system, but also by the weather itself. Here, we propose the use of the odds ratio (benefit), ORB, as a measure, which is not influenced by the base-rate of the event and thus enables a fair comparison of categorical forecasts for different years, regions, events, etc. The ORB has a simple interpretation and it permits a split of forecasting skill into contributions from forecasting the event and the non-event. Applying this measure to operational quantitative precipitation forecasts reveals that forecasts of more extreme (rare) events have more skill than forecasts for more ‘normal’ events which is contrary to the results typically obtained with other categorical measures traditionally used in meteorology and also to subjective perception. Both of the latter can be interpreted as a delusive consequence of the ‘neglect of the base-rate’ effect. Further consequences are described for the composition of model trials and for the verification of forecast warnings. Over recent years there have been trends showing a small improvement in skill and a large reduction in model bias for forecasts of slight precipitation.