A multivariate approach to model skill assessment

Although the purpose of models is to simplify complex reality to allow the investigation of patterns, processes and relationships, many ecosystem models retain high levels of complexity. The outputs from such models are highly multivariate. Taking the view that a perfect model simulation of a spatial domain over a determinate time period will reproduce observed variables from the same place over the same period perfectly, we demonstrate how appropriate multivariate methods may be used to elucidate patterns within observations and model outputs, to compare patterns between them, and to explore the nature and spatio-temporal distribution of model errors. Analyses based on observations collected from the southern North Sea in 1988-89 are compared to analyses based on an equivalent dataset extracted from the output of the POLCOMS-ERSEM model. A combination of PCA and nonparametric multivariate approaches is used to demonstrate that in broad terms the model performs well, simulating patterns in, and interrelationships between, a range of variables. Errors are greatest in late winter and early spring, and are associated with inaccurate estimation of the magnitude of primary production in coastal waters and the amount of suspended particulate matter in the water column. (C) 2008 Elsevier B.V. All rights reserved.