The extreme dependency score: a non-vanishing measure for forecasts of rare events

Accurate prediction of rare high-impact events represents a major challenge for weather and climate forecasting. Assessment of the skill at forecasting such events is problematic because of the rarity of such events. Skill scores traditionally used to verify deterministic forecasts of rare binary events, such as the equitable threat score (ETS), have the disadvantage that they tend to zero for vanishingly rare events. This creates the misleading impression that rare events cannot be skilfully forecast no matter which forecasting system is used. This study presents a simple model for rare binary-event forecasts and uses it to demonstrate the trivial non-informative limit behaviour of several often-used scores such as ETS. The extreme dependency score (EDS) is proposed as a more informative alternative for the assessment of skill in deterministic forecasts of rare events. The EDS has the advantage that it can converge to different values for different forecasting systems and furthermore it does not explicitly depend upon the bias of the forecasting system. The concepts and scores are demonstrated using an example of 6-hourly precipitation total Met Office forecasts for Eskdalemuir in Scotland over the period 1998 - 2003. Copyright (c) 2008 Royal Meteorological Society; (c) Crown Copyright 2008. Reproduced with the permission of the Controller of HMSO. Published by John Wiley & Sons, Ltd.