Use of objective criteria for the assessment of biogeochemical ecosystem models

Ecosystem modeling is confronted with complex biological systems and changing environmental conditions. A model which describes ecosystem behavior under all conditions has not been found yet and there does not exist one 'true' model for a specific ecosystem. Often ecosystem models describe the measured data more or less well, but most judging criteria for model performance are rather subjective. Furthermore. from a mathematical view point the calibrations of ecosystem models are hardly ever unique. The aim of this stud; was to develop and use criteria which permit an objective comparison of different models to the observed field data and to each other. A given model which describes a specific system significantly better will be declared the 'valid' model while the other will be rejected. The term 'valid' is used here in a sense that any model that could not be proven invalid would be a valid model for the system. We used the biogeochemical soil models MAGIC (Cosby, B.J., Hornberger, G.M., Wright, R.F., 1985. Modelling the effects of acid deposition: assessment of a lumped-parameter model of soil water and stream water chemistry. Water Resour. Res. 21, 51-63) and the SO-Model (derived from the Batch Equilibrium Model (BEM; Prenzel, J., 1991. Introduction to BEM (Batch Equilibrium Model), vol 28. Berichte des Forschungszentrums Waldokosysteme/Waldsterben, Gottingen, 51 pp.). The data set used was the soil solution chemistry in a forest ecosystem of the Selling area (North-West Germany). To test the performance of the models four criteria were used: the efficiency (Martinec, J., Range, A., 1989. Merits of statistical criteria for the performance of hydrologic models. Water Resour. Bull. 25 (2), 421-432; Hinzman, L.D., Kane, D.L., 1991. Snow hydrology of a headwater artic basin; 2. Conceptual analysis and computer modelling. Water Resour. Res. 27, 1111-1121), the Normalized Mean Absolute Error (NMAE, given by Janssen, P.H.M., Heuberger, P.S.C., 1995. Calibration of process orientated models. In: van Grinvsen. J.J.M. (Ed.) Modelling Water; Carbon and Nutrient Cycles in Forests: Application of 16 Simulation Models to a Spruce Stand at Selling, Germany. Ecological Modelling, vol. 83, pp. 55-66), the confidence interval test (CIT, developed in this study) and the model rejection criteria (Sun, N.Z., 1994. Inverse Problems in Groundwater Modelling. Dordrecht, 337 pp.). Whereas the efficiency and NMAE are related to the averaged data, the CIT and the model rejection criteria include the spatial heterogeneity at every time step. When evaluated visually, both model results might be accepted. From the application of the model performance criteria we selected the MAGIC model as the 'valid' model for our system. (C) 1998 Elsevier Science B.V. All rights reserved.