A proposal of an indicator for quantifying model robustness based on the relationship between variability of errors and of explored conditions

The evaluation of biophysical models is usually carried out by estimating the agreement between measured and simulated data and, more rarely, by using indices for other aspects, like model complexity and overparameterization. In spite of the importance of model robustness, especially for large area applications, no proposals for its quantification are available. In this paper, we would like to open a discussion on this issue, proposing a first approach for a quantification of robustness based on the variability of model error to variability of explored conditions ratio. We used modelling efficiency (EF) for quantifying error in model predictions and a normalized agrometeorological index (SAM) based on cumulated rainfall and reference evapotranspiration to characterize the conditions of application. Population standard deviations of EF and SAM were used to quantify their variability. The indicator was tested for models estimating meteorological variables and crop state variables. The values provided by the robustness indicator (I-R) were discussed according to the models' features and to the typology and number of processes simulated. I-R increased with the number of processes simulated and, within the same typology of model, with the degree of overparameterization. No correlation were found between I-R and two of the most used indices of model error (RRMSE, EF). This supports its inclusion in integrated systems for model evaluation. (C) 2009 Elsevier B.V. All rights reserved.